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the  stimulus  fundamental  or  2nd  harmonic  MF.  The  CEPs  were  characterized  in  terms 
of  their  magnitude,  phase  and  latency  Modulation  Transfer  Functions,  background  EEG, 
repeatability,  linearity,  and  stimulus  intensity  and  subject  attentiveness 
dependencies. 

CEP  magnitude  peeks  were  obtained  at  5-6  Hz  (major),  8-14  Hz,  18-22  Hz  and  40-50 
Hz  (only  auditory)  MF  regions.  Estimated  from  linear  models  fitted  to  the  magnitude 
data,  these  peeks  were  associated  with  averaged  latency/delay  values  of  205  msec/60 
msec,  200  msec/80  msec,  70  msec/30  msec  and  25  msec/10  msec,  respectively,  suggesting 
cortical  origins.  Most  of  the  results  were  derived  from  two  MF  regions,  "Beta"  (16-25 
Hz)  and  "Theta"  (5-6  Hz).  Responses  elicited  from  these  MF  regions  were  consistent, 
exclusively  evoked,  repeatable  and  easily  detectable.  In  comparison  to  the  auditory 
CEPs,  the  visual  CEPs  were  associated  with  higher  signal/noise  magnitude  ratio,  higher 
inter-subject  variability  and  lower  Intra-subject  variability. 

Genuine  AxV  effects  of  5.5  dB  magnitude-gain  and  40  degree  phase-shift  were 
found.  Magnitude  inhibition  and  phase  lag  were  generally  associated  with,  and  induced 
by,  the  short-latency  Beta  MF  region.  Magnitude  data  was  primarily  affect  by  "Internal" 
parameters  (intra-modality  stimulus  intensity  or  MF)  and  phase  data  by  "External" 
parameters  ( cross -modal ity  stimulus  Intensity).  Attention  allocation  did  not  play  a 
major  role. 

This  line  of  investigation  can  lead  to  further  understanding  of  human  sensory- 
channel  transfer  function  and  sensory  information  integration,  and,  may  help  in  design¬ 
ing  more  effective  man-machine  communication  channels. 
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Part  I:  Introduction 


i 


Chapter  1 
Background 


1.1  Motivation 

The  motivation  for  this  research  was  to  develop  an  Event  Related  Cortical  Potential 
(ERCP)  methodology  for  non-invasivcly  monitoring  the  auditory  and  visual  sensory 
channel  engagement  and  interaction  (AxV)  in  humans.  Sensory  interaction  is  inferred 
whenever  response  variations  of  one  sensory  channel  can  be  attributed  to  parameter 
changes  of  an  additional  stimulus,  simultaneously  presented  to  another  sensory  channel. 
This  type  of  on-line  continuous  monitoring  and  parameter- space  characterization  of  the 
AxV  process  has  never  been  attempted  before.  This  approach  can  be  utilized  in  many 
existing  disciplines  and  promote  new  research  directions:  providing  further  understand¬ 
ing  of  human  sensory  channel  transfer  functions  and  sensory  information  integration, 
investigating  the  unaccounted  for  intra-subject  unisensory  ERCP  variability  by  applying 
an  exhaustive  multisensory  input  stimulus,  facilitating  the  effectiveness  of  audio-visual 
trainers,  monitoring  sensory  channel  engagement  and  alerting  against  overloading  tasks 
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that  might  deteriorate  human-operator  performance,  and  designing  more  effective  and 
efficient  man-machine  communication  channels. 

1.2  Brain  potentials 

1.2.1  EEG  and  evoked  potentials 

The  human  brain  and  its  marvelous  complexity  is  regarded  by  many  scientists  as  the 
"last  frontier".  Although  extensively  investigated,  there  is  the  valid  reservation  of 
whether  the  human  brain  can  ever  reveal  itself.  Ever  advancing  technology  often  pro¬ 
vides  analogies  and  metaphors  to  theorize  how  the  brain  "works",  i.e.  "holographic"  type 
memory,  "distributed-computational  network"  type  cognitive  information  processing, 
etc.  The  research  described  in  this  study  also  incorporates  metaphors,  terminology  and 
analysis  techniques  common  in  engineering  and  computer  science  fields.  Although  never 
completely  valid  or  accurate,  these  "mental  tools"  can  simplify  problems  and  facilitate 
intuitive  understanding  of  their  solutions. 

One  obvious  function  of  the  human  brain  is  encoding  and  processing  environmental 
information.  External  cues  (detectable  features  in  our  environment)  undergo  energy 
transformation  at  the  receptor  level  and  their  neural  codes  are  deployed  and  mediated 
to  the  cortical  level  through  specific  and  diffused  sensory  channels.  Each  specific  sen¬ 
sory  channel  is  a  neural  pathway  comprised  of  a  chain  of  neuronal  aggregates.  The 
contact  between  two  neurons  is  accomplished  mostly  through  chemical-coupled 
synapses.  The  informational  "bit"  passed  through  a  synaptic  gap  is  believed  to  be  the 
Action-Potential  (AP),  an  electrical  biphase  pulse  (1-2  msec  duration)  capable  of  prop¬ 
agating  along  the  neuron  axon.  A  synchronized  response  from  a  group  of  excited 
neurons  is  referred  to  as  a  compound-potential.  With  bipolar  far-field  recording  (dif¬ 
ferential  amplification  of  scalp  potentials),  the  compound-potentials  reflect  the  dipole- 


equivalent  model  of  either  the  propagating  volley  of  APs  or  the  post-synaptic  potentials 
of  the  targeted  neuronal  population  (Childers,  1977;  Vaughan,  1982).  The  cortical  pyr¬ 
amidal  cell  layer  is  a  good  example  of  the  latter.  When  neurons  in  this  homogenously- 
architectured  layer  are  synchronously  activated,  the  far-field  potential  gradient  across 
this  layer  can  be  differentially  recorded  from  the  scalp  (Vaughan,  1982).  The  magnitude 
of  this  response  is  a  function  of  the  source  strength,  distance  and  orientations  relative 
to  the  recording  electrode  sites. 

The  brain  is  never  "quiet*.  Even  without  any  apparent  input,  an  on-going  electrical 
activity,  termed  background  EEG  (electroencephalogram)  is  always  present.  This 
background  activity  has  been  related  to  many  processes  (cortical  activation  level,  system 
"noise-lever,  cognitive  information  processing  indicator,  etc.)  and  found  to  be  affected 
by  many  behavioral  and  physiological  factors.  When  an  input  stimulus  is  presented,  a 
different,  stimulus-related  Evoked  Potential  (EP)  can  be  recorded.  Scalp  potential  dis¬ 
tributions  and  frequency  response  comparisons  of  background  EEG  and  EP  signals  re¬ 
veal  that  different  sources  (or  subsystems)  contribute  to  each  signal  (Childers  and  Perry, 
1971). 

It  will  be  helpful  to  pause  here  and  formally  define  and  characterize  the  ERCP  signal. 
ERCP  is  the  total  causal  brain  electrical  activity,  mostly  from  cortical  origins,  which 
follows  a  defined  administered  stimulus.  It  can  be  recorded  non-invasively  in  response 
to  auditory,  visual  or  somatosensory  stimuli.  The  stimulus  can  be  transient  (click,  flash) 
or  continuous  (sinusoidal  or  sinusoidally  modulated  carrier  signal)  and  the  ERCP  data 
can  be  processed  both  in  time  and  frequency  domains.  In  most  cases,  the  input  stimulus 
can  be  precisely  controlled  and  characterized,  or  in  other  words,  a  defined  input  feature 
vector  can  be  constructed. 
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Some  of  the  more  important  features  of  a  single  ERCP  are: 


1.  It  has  small  magnitudes  (0.5-50  microvolt)  with  low  Signal/Noise  Ratio  (SNR); 

2.  Noise  (EEC,  EM G,  ECG,  instrumentational  artifacts,  etc.)  and  ERCP  signals  fre¬ 
quently  share  a  common  frequency  band; 

3.  It  is  a  highly  variable  compound-response  (produced  by  an  unlikely  linear,  time- 
variant,  complex  neural  system); 

4.  It  reflects  both  the  system  “hardware"  (structural  response)  and  "software"  (cogni¬ 
tive  response);  the  latter  is  more  likely  to  be  detected  in  longer  latency  ERCP  com¬ 
ponents. 

Apart  from  the  single-trial  ERCP  methodology  using  a  pattern  recognition  approach 
(Vidal,  1977;  Horst  and  Donchin,  1980),  several  methods,  from  averaging  and  corre¬ 
lation  methods  to  special  adaptive  filtering  methods,  have  been  employed  to  optimally 
extract  the  ERCP  (Aunon  and  Sencaj,  1978;  Sencaj  ct  al.  1979). 

1.2.2  Evoked  potential  methodologies 

EP  methodologies  were  developed  under  subjective  ad-hoc  hypotheses  with  various 
intended  applications  in  mind.  These  methods  can  be  ranked  and  compared  by  the  fol¬ 
lowing  criteria:  neural  response  acquisition-time,  size  of  the  targeted  or  responding 
neuronal  population  (observation  unit)  and  degree  of  observation  unit  spatial  distrib¬ 
ution  disclosed  and  displayed.  For  the  purpose  of  example  and  reference,  the  method¬ 
ology  of  traditional  ensemble  averaging  of  Transient  Evoked  Potentials  (TEPs)  is 
reviewed  in  the  following  paragraphs. 


I 
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1.2. 2.1  Ensemble  averaging  approach 


The  major  assumption  behind  ensemble  averaging  is  that  identical  stimulus  vectors 
under  the  same  controllable  experimental  conditions  are  translated  through  the  same 
neural  pathways  in  the  same  manner,  yielding  exactly  the  same  spatial  and  temporal 
distribution  of  the  ERCP.  In  other  words,  the  sensory  channel  is  treated  as  a  time- 
invariant  system,  triggered  by  an  adequate  transient  signal.  The  same  type  of  reasoning 
also  linked  the  ERCP  to  the  associated  behavioral  observation.  As  the  ERCP  is  "time- 
locked'  to  the  input  stimulus,  so  is  the  behavioral  pattern  "feature-locked"  to  the  ERCP. 
This  conceptual  approach  provided  the  rationale  for  studies  in  which  the  expected  future 
behavior  was  extrapolated  from  the  ERCP,  or  the  unknown  past  input  stimulus  vector 
was  interpolated  from  the  present  behavior  (Callaway  et  al.,  1978;  Begleiter,  1979).  The 
descriptive  ERCP  features  used  in  these  studies  were:  means  and  variances  of  ampli¬ 
tudes  and  latencies,  cross-correlations,  wave  areas,  principal  components,  power  spec- 
trams  and  others. 

Most  of  the  ERCP  studies  investigated  the  sensory  channel  response  to  a  limited, 
single  modality  stimulus  vector  (Callaway  et  al,  1978).  Inter-  and  intra-subject  ERCP 
variability  were  often  cited  as  the  major  obstacles  which  prevented  more  significant  ties 
between  the  ERCP  features  and  the  input  stimulus  vector.  Treating  the  brain  as  a 
time-variant  system  and  pre-processing  the  whole  ERCPs  ensemble  (posterior  adaptive 
filtering  preceding  the  averaging),  the  "best'  representative  ERCP,  free  of  temporal 
'jitters'  in  its  components,  was  derived  (de  Weerd,  1981;  de  Weerd  and  Kap,  1981).  The 
advantages  of  this  method  are  obvious,  as  are  the  drawbacks.  A  computer  is  needed  for 
data  storage,  digital  filtering  and  processing,  and  more  important,  the  resultant  ERCP 
lacks  valuable  information  regarding  the  dynamics  of  the  ERCP  components  throughout 
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the  recording  session.  In  simple  ensemble  averaging,  described  previously,  this  infor¬ 
mation  is  partially  expressed  by  the  ERCP  variance. 

In  summary  ,  ensemble  averaging  methodology  utilizes  long  acquisition-times  (in  or¬ 
der  of  minutes)  to  obtain  a  limited  average  potential  distribution  of  ERCPs,  probably 
generated  in  large  and  distributed  neuronal  populations. 

1. 2.2.2  Single-trial  approach 

A  faster  acquisition-time  method  must  be  used  to  allow  detection  and  faithful  moni¬ 
toring  of  parameter -changes  in  smaller  Time-Constant  (TC)  EPs  (produced  by  a  sensory 
channel  with  a  wide  frequency  response  bandwidth).  This  can  be  implemented  by 
dichotomizing  key  features  of  the  stimulus  (digitizing  the  selected  features),  and/or  by 
restricting  its  response  interpretation  (collapsing  the  EP  data-base).  These  principles 
have  been  utilized  in  many  single-trial  EP  studies,  where  pattern-recognition  algorithms 
and  other  statistical  analysis  techniques  were  employed  to  detect  gross  features  within 
an  EP  generated  for  a  single  stimulus  (Squires  and  Donchin,  1976;  Vidal,  1977;  Gevins, 
1980;  Childers  et  a!.,  1982;  McGillem  and  Aunon,  1977,  1983,  1985).  Vidal  (1977)  was 
able  to  correctly  classify  single-trial  TEPs  generated  by  high  feature-contrast  stimuli 
(stimulating  different  halves  of  the  retina).  This  conceptual  approach  necessitates  an 
exhaustively  defined  and  controlled  input  stimulus  and  utilizes  adaptive  signal-extraction 
techniques  to  cope  with  the  intra-subject  variability. 

1. 2.2.3  Lock-in  approach 

Other  implementations  of  these  principles  are  the  use  of  phase-lock  techniques  (also 
referred  to  as  complex  or  synchronous  demodulation,  lock-in,  or  Fourier  analysis  tech¬ 
niques)  in  recording  Continuous  Evoked  Potentials  (CEPs)  (Regan,  1966;  Levine  et  al., 
1972).  When  a  target  feature  within  the  input  vector  stimulus  is  periodically  modulated 
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(intensity  modulation,  phase  modulation,  etc.),  a  CEP  vector  component  phase-locked 
to  the  Modulation  Frequency  (MF)  can  be  tracked  and  monitored  with  recording  TCs 
of  3-30  sec.  Generally,  this  technique  provides  faster  EP  acquisition-time  and  better 
SNR  enhancement  than  do  the  ensemble  averaging  and  single-trial  techniques,  respec¬ 
tively.  The  narrower  the  Bandwidth  (BW)  of  the  "reacting"  MFs  (MFs  for  which  the 
EP  data  validates  or  negates  the  research  hypotheses),  the  more  efTicient  this  technique 
becomes.  The  drawback,  however,  is  the  loss  of  direct  EP  temporal  information,  im¬ 
plicitly  obtained  in  single-trial  or  ensemble  averaging  techniques. 

In  light  of  the  above,  the  phase-lock  technique  is  suitable  for  the  purpose  of  contin¬ 
uously  monitoring  the  sensory  channel  engagement.  CEPs  can  be  simultaneously  re¬ 
corded  from  one  or  more  sensory  channels  under  unimodal  or  multimodal  stimulus 
conditions.  Due  to  the  high  frequency  resolution  of  the  phase-lock  technique,  the  effects 
of  cross-modal,  similar  MF  stimuli  can  be  investigated.  The  following  section  provides 
the  rationale  for  investigating  such  effects,  specifically  between  the  auditory  and  visual 
sensory  channels. 

1.3  Auditory-visual  interaction 

1.3.1  Supporting  behavioral  phenomena 

It  is  well  known  that  auditory  and  visual  information  supplement  and  complement 
each  other  in  daily  human  communications.  Speech  intelligibility  was  greatly  enhanced 
when  supplemented  by  visual  cues  from  a  speaker's  lip  movements  (Sumby  and  Pollack, 
1954).  Also,  visual  recognition  of  a  word  was  enhanced  when  presented  simultaneously 
with  a  matched  spoken  word  (Smith,  1965).  On  the  other  hand,  early  auditory  depri¬ 
vation  in  children  impaired  the  temporal  pattern-reproduction  ability  in  auditory,  and 
visual  tasks  as  well  (Sterrit  et  a!.,  1966).  These  three  examples  help  to  substantiate  an 
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intuitive  notion  about  the  Auditory-Visual  interaction  (AxV)  mechanism  in  humans  and 
its  importance. 

There  is  some  evidence  that  the  auditory  input  in  AxV  tasks  is  dominant.  Auditory 
intermittent  stimulation  drove  the  perceived  visual  stimulus  flicker  rate,  making  it  almost 
independent  of  the  actual  visual  flash  rate.  The  visual  flicker  rate,  on  the  other  hand, 
could  not  drive  the  perceived  auditory  flutter  rate  (Knox,  1945).  Auditory  flutter  in¬ 
creased  the  Critical  Flicker  Frequency  (CFF)  (Oglive,  1965)  and  it  was  much  harder  to 
match  with  a  stable  flicker  rate  than  vice  versa  (Gebhard  and  Mowbray,  1959).  Sensi¬ 
tivity  indices  (differences  between  expected  values  of  the  responses  to  noise  alone  and 
noise  +  signal)  and  Reaction  Time  (RT)  tests  consistently  showed  that  the  detection  of 
a  bimodal  signal  was  better  than  the  auditory  signal  alone,  which  was  better  than  the 
visual  signal  alone  (Loveless,  et  al.,  1970). 

Some  experiments  have  demonstrated  the  specific  nature  of  the  AxV  phenomenon. 
Maruvama  (1959)  investigated  the  effects  of  tones  with  different  frequencies  and  inten¬ 
sities  on  the  absolute  visual  threshold  at  various  times  during  a  4  second  period  of 
auditory  stimulation.  For  foveal  vision,  the  threshold  shift  effect  was  immediate,  lasted 
for  3  seconds,  and  was  proportional  to  the  tone's  intensity  level.  For  peripheral  vision, 
the  peak  effect  was  obtained  about  2  seconds  after  tone  onset.  High  frequency  tones 
facilitated  the  effect  while  lower  frequency  tones  (below  50-100  Hz)  inhibited  it.  Further, 
Maruvama  (1961)  demonstrated  that  stimulating  the  right  peripheral  vision  increased  the 
sensitivity  of  the  left  ear  only,  and  when  that  ear  was  stimulated,  a  contralateral  effect 
on  half  of  the  visual  field  sensitivity  was  observed. 


1.3.2  Possible  neural  basis 


As  previously  indicated,  language  and  speech  ability  in  humans  requires  bisensory 
convergence  of  auditory  and  visual  information.  Human  neuroanatomy  studies  provide 
structural  evidence  about  the  auditory  and  visual  afTercnt  systems  converging  in  points 
along  fast-conducting  channels  (Colliculi,  Geniculate  Body,  Thalamus  and  associated 
non-specific  cortical  areas,  etc.)  as  well  as  onto  diffused,  slow-conducting  neuronal 
masses  (Reticular  Formation,  etc.). 

Walter  (1964)  investigated  the  convergence  of  auditory,  visual  and  tactile  responses 
in  the  human  non-specific  cortex,  recorded  both  from  implanted  microelcctrodcs  and 
surface  scalp  electrodes.  Habituation  characteristics  and  differences  inflicted  by  the  re¬ 
cording  techniques,  led  him  to  conclude  that  scalp-recorded  EPs  were  comprised  of  spe¬ 
cific  and  nonspecific  responses,  the  latter  predominant.  Specific  and  non-specific 
responses  originated  from  the  primary  and  the  associative  cortical  areas,  respectively. 
He  found  that  signals  in  all  studied  modalities  converged  at  the  frontal  cortex  (a  non¬ 
specific  area)  and  were  widely  dispersed  therein.  Intracranial  recording  from  the  same 
sites  showed  that  each  modality  had  its  own  "signature".  Distinguished- featured, 
modality-dependent  auditory  and  visual  EPs  were  recorded  from  this  area  with  latencies 
of  25  and  35  msec,  respectively. 

The  convergence  of  sensory  inputs  was  demonstrated  also  at  longer  latency  re¬ 
sponses.  Davis  ct  al.  (1972)  recorded  TEPs  in  response  to  "comfortably  strong"  and 
approximately  equal  subjective  magnitude  audio,  tactile,  shock  and  flash  stimuli.  Similar 
latencies,  waveforms,  amplitudes  and  recovery  periods  of  these  so-called  "vertex- 
potentials"  (recorded  from  Cz-M  1/Forchcad  with  a  1-15  Hz  BW  amplifier)  were  found 
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in  all  modalities,  in  summary,  it  has  been  suggested  that  the  auditory  and  visual  systems 
have,  or  share,  common  "hardware"  facilities  or  coding  mechanisms. 

Using  a  "holistic"  approach  in  analyzing  the  sensory  neural  codes  in  feature  ex¬ 
traction,  Erickson  (1974)  suggested  that: 

The  neural  possibilities  afforded  to  one  sense  by  the  structure  and  physiology  of 
its  neurons  and  their  connections  are  also  available  to  other  senses,  and  thus  the 
encoding  parameters  available  to  one  are  also,  to  some  degree,  available  to  an¬ 
other. 

Using  solely  the  neuronal  activity  attributes  of  quality,  intensity  and  duration  formulated 
in  the  nineteenth  century,  Erickson  defined  classes  of  change  in  neural  activity  related 
to  these  attributes.  Intensity  was  coded  by  change  in  the  amount  of  neural  activity 
without  change  in  the  appearance  of  this  activity,  while  quality  (i.e.  sound  spectrum, 
object  position  in  space,  color,  etc.)  was  coded  by  change  in  the  appearance  of  the  neural 
activity  (shifting  to  a  new  population  of  neurons  or  an  across-fiber  pattern  change  of 
activity  within  the  same  neuronal  population).  In  this  respect,  an  Amplitude  Modulated 
(AM)  tone  was  comparable  to  an  AM  spotlight  (intensity  modulation  of  a  quality  fea¬ 
ture)  and  a  Frequency  Modulated  (EM)  tone  was  comparable  to  the  position  modu¬ 
lation  of  the  spotlight  (quality  modulation  in  a  given  intensity). 

The  quality  attribute  was  coded  topographically  (spatial  position  in  the  retina,  as¬ 
cending  tracks,  and  the  primary  cortex)  by  narrowly  tuned  neurons  or  non- 
topographically  (color  coding)  by  a  smaller  population  of  broadly  tuned  neurons. 
Erickson  suggested  that, 

Each  neuron  may  be  sensitive  along  many  broadly  tuned  (non-topograpliic)  di¬ 
mensions  but  sensitive  to  only  one  narrowly  tuned  (topographic)  dimension, 

which  is  in  accordance  with  the  conflicting  principles  of  necessary  neural  redundancy 

and  neural  volume  reduction. 
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A  plausible  interpolation  of  Erickson's  analysis  is  that  for  a  given  quality,  the  inten¬ 
sity  and  quality  related  neuronal  activities  (or  populations)  are  orthogonal.  So,  a  nar¬ 
rowly  tuned,  topographically-arranged  neuronal  sub-population,  engaged  in  a  specific 
quality  feature  extraction,  may  at  the  same  time  participate  in  other,  very  broadly  tuned, 
diffused  or  distributed  neuronal  populations  encoding  intensity  of  other  different  quali¬ 
ties.  Another  possibility  is  that  a  very  broadly  tuned  neuronal  population  participate  in 
intensity  coding  of  more  than  one  quality  feature. 

Direct  EP  evidence  regarding  the  auditory-visual  bisensory  interaction  is  provided  in 
the  introduction  section  of  the  AxV  experiment  in  this  study. 
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Chapter  2 

Research  Objectives 


The  purpose  of  this  investigatory  study  was  to  adapt  the  phase-lock  methodology  for 
recording  and  characterizing  Continuous  Evoked  Potential  (CEP)  variations  induced  by 
the  Auditory-Visual  bisensory  interaction  process  (AxV).  The  specific  AxV  aspect  in¬ 
vestigated  here  was  the  dependency  of  single  sensory  channel  CEP  variations  on  cross- 
or  inter-modality  stimulus  parameters.  The  research  goal,  obtaining  the  effective  AxV 
stimulus  parameter-space  for  future  investigation,  was  translated  into  the  following  ob¬ 
jectives: 

1.  Developing  the  hardware-system  needed  for  continuous  generation,  recording  and 
processing  of  the  CEPs;  Including:  unimodal  and  bimodal  stimulus  generation  with 
programmable  gate  control,  sensitive  and  stable  acquisition  of  CEPs  over  long  re¬ 
cording  sessions  and  providing  for  on-line  and  off-line  data  processing. 

2.  Obtaining  and  optimizing  on-line  detection  of  auditory  and  visual  CEPs;  Exploring 
stimulus  conditions,  experiment  paradigms,  recording  sites  and  CEP  processing. 
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3.  Characterizing  the  CEPs  in  terms  of  their  magnitude,  phase,  estimated  latency  and 
delay,  2nd  harmonic  component,  response  stability  and  the  e/Tccts  of  stimulus  in¬ 
tensity  and  subject  attentiveness. 

4.  Demonstrating  an  AxV  process  on  a  selected  stimulus  parameter-space;  CEP  mag¬ 
nitude  and  phase  variations  were  associated  with  specific  stimulus  conditions  (mod¬ 
ulation  frequencies,  intensities  and  attention  allocation). 
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Chapter  3 
Assessment 

3.1  Outline  of  experiment 

The  main  hypothesis  of  this  research  was  studied  in  the  AxV  experiment,  where  at¬ 
tention,  stimulus  modalities  and  intensity  effects  on  magnitude  and  phase  of  the  Con¬ 
tinuous  Evoked  Potentials  (CEPs)  were  investigated.  All  preceding  experiments  were 
designed  to  determine  optimal  values  for  the  Independent  Variables  (I Vs)  or  provide 
controls  for  the  Auditory-Visual  Interaction  (AxV)  experiment.  Results  from  the  MTF 
experiment  were  used  to  derive  5-61  Hz  range,  magnitude,  phase  and  latency  Modu¬ 
lation  Transfer  Functions  (MTFs).  Best  Modulation  Frequency  (MF)  regions,  excluding 
the  Alpha  EEG  peak  activity  frequency  regions,  were  selected  for  further  detailed  inves¬ 
tigation.  Results  from  the  following  DISC  and  I  NT  experiments  facilitated  choice  of 
optimal  M  F  values.  Based  on  response  magnitude  and  stability  characteristics,  two  op¬ 
timal  MFs  (from  Theta  and  Beta  MF  regions)  were  chosen  for  each  subject  to  be  used 
in  the  AxV  experiment.  Utilizing  the  IVs  selected  in  the  DISC  and  INT  experiments, 
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CEP  magnitude  and  phase  dependencies  on  subject  attentiveness  under  unimodal  stim¬ 
ulation  conditions  were  studied  in  the  ATT  experiment. 


3.2  Working  hypotheses 

The  feasibility  of  detecting  CEP  variations  induced  by  the  AxV  process  depends  on 

the  validity  of  the  following  ad-hoc  assumptions: 

1.  Integration  of  auditory  and  visual  information  is  partially  "hardwared".  The  two 
inputs  physically  converge  upon  a  common,  multimodal  neuronal  population,  rather 
than  being  just  "observed"  and  merged  by  cognitive  "software". 

2.  The  AxV  effects  are  consistent  and  durable,  at  least  for  the  duration  of  the  bisensory 
stimulations  used  in  this  study. 

3.  The  AxV  process  afreets  the  compound  potentials  (or  EPs)  generated  by  the  multi¬ 
modal  neuronal  population. 

4.  The  ratio  of  the  polysensory  to  unisensory  neuronal  population  is  quantitatively 
significant  to  allow  detection  of  the  AxV  effect. 

5.  Common,  cross-subject  general  AxV  trends  can  be  observed  (tolerable  low  inter- 
subject  variability). 

3.3  Common  resources 

3.3.1  Subject  participation 

Ten  students,  all  reporting  normal  hearing  and  normal  or  corrected  vision,  partic¬ 
ipated  in  this  research.  Description  of  these  subjects  is  presented  in  TABLE  I  3.1  (no¬ 
tation:  TABLE/FIGURE  <  part  I,  II  or  III  >  <  chapter  1-10  >.<  running  number  • ). 
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One  subject,  distinguished  by  a  very  poor  raw  CEP  data  Signal/N'oise  Ratio  (SNR), 
could  not  be  processed  and  was  excluded  from  further  statistical  analysis. 


SUBJECT  DESCRIPTION 

TABLE  I  3.1  :  All  subjects  were  students  at  the  University  of  Connecticut.  Nota¬ 
tion  :  1st  TABLE  in  chapter  3  in  part  I,  N  :  normal,  C  :  corrected  to  normal,  P  : 
paid,  V  :  volunteer. 


| SUBJECT | AGE 

|  | years 

1  1 

|  SEX  | 

1 M/F  | 

HANDED 

R/L 

| VISION 

1  N/C 

INCENTIVE 

V/P 

|DATA  USEABLE | 

|  yes/no  | 

l 

|  AM 

1 

1  25 

I 

1  F  j 

1  1 

R 

|  N 

V 

1  1 

1  yes  | 

l  I 

|  CR 

1 

1  21 

J 

i  i 

1  F  | 

1  1 

R 

1  N 

P 

1  1 

1  yes  | 

1  1 

1  EW 

1  26 

1 

1  F  | 

1  | 

R 

i  N 

V 

1  yes  1 

1  1 

|  KC 

1  27 

1 

1  1 

1  F  | 

R 

j  c 

1 

V 

1  1 

1  yes  1 

1  1 

|  MR 

1 

|  22 

1 

1  i 

1  M  | 

R 

1 

1  N 

P 

1  1 

1  yes  1 

1  1 

|  PM 

1  21 

1 

1  1 

1  M  | 

i  i 

R 

1  N 

P 

1  1 

1  yes  1 

1  1 

1  RT 

1 

|  20 

1 

1  1 

1  M  | 

l  i 

R 

1  N 

P 

1  1 

1  ^  -s  1 

|  SC 

i 

|  22 

I 

i  i 

1  F  | 

1  | 

L 

1  c 

P 

•  1 

1  yes  | 

1  1 

|  TN 

t 

|  30 

1 

1  1 

1  M  | 

1  | 

R 

1  1 

V 

!  1 

1  yes  | 

l  I 

1  fl 

1  27 

1  1 

1  M  | 

R 

j  C 

V 

1  1 

1  no  | 

For  most  of  the  subjects,  the  data  was  collected  in  three  recording  sessions,  about 
four  to  five  hours  each,  over  a  period  of  two  to  three  weeks.  Sessions  and  experiments 
were  standardized  through  subject  conditioning  (acquaintance,  information,  motivation) 
and  experiment  procedures  (instructions,  involvement,  trials,  breaks).  The  research  was 
presented  as  a  Reaction  Time  (RT)  study  and  subjects  were  not  disclosed  the  real  goals 
of  the  experiment  nor  the  hypotheses  involved,  prior  to  completing  their  research  en¬ 
gagement. 


-17- 


3.3.2  Research  system  hardware  description 

3. 3. 2.1  Stimulus  generation 

A  program  selector  module  ("Home  Made",  HM)  drove  and  controlled  a  digital  sum 
of  sines  synthesized  module  (HM),  producing  pre-programmed,  experiment-dependent 
stimulus  (AS1G,  VSIG),  reference  (AREF,  VREF)  and  control  (AGATE,  VGATE) 
signals  (FIGURE  I  3.1).  GATE  signals,  reflecting  stimulus  "on"  and  "off'  conditions, 
were  digital  (TTL)  signals.  All  other  signals  were  digital-to-analog  converted  (DAC), 
filtered  sinusoidal,  or  sum  of  sinusoidal  signals  (1  RMS  volt  per  sine  waveform)  with  a 
2nd  harmonic  attenuation  better  than  -70  dB).  The  continuous  REF  and  gated  S1G 
signals  were  the  modulation  frequencies  and  modulated  carriers  (915  Hz  for  audio  stim¬ 
uli  and  0  Hz  for  visual  stimuli),  respectively.  AS1G  and  VSIG  signals  were  transferred 
through  Logarithmic  Attenuator  and  Power  Driver  modules  (HM)  and  converted  to 
sound  and  light  stimuli  by  a  headphone  set  (TELEPHONICS,  TDH-49P)  and  a  Glow- 
tube  (METCOM,  tube  #7920253),  respectively.  The  physical  stimulus  2nd  harmonic 
attenuation  was  better  than  -60  dB. 

3. 3. 2. 2  Data  recording  and  processing 

The  EP  or  EEG  signals  were  differentially  amplified  and  bandpassed  between  0.5-570 
Hz  using  first-order,  cascaded  High-Pass  and  Low-Pass  Filters  (HPF-LPF).  The  front- 
end  in  each  of  the  Differential  Amplifiers  (DA,  HM  modules)  was  a  Medical  Isolation 
Amplifier  (INTRONICS,  1A-297  or  IA-296),  characterized  by  high  Common  Mode 
Rejection  Ratio  (CMRR  >  120  dB)  and  input  impedance  (>  100M  ohms),  low  input 
noise  ( <  1.5  microvolt)  and  a  driven  reference. 
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THE  RESEARCH  SYSTEM 


HONrmU.  1100 


FR  INC  IT  OK  AJ»PLXO  USSAJtCH 
ECdG  Hod.:  5  20t 


HTWIXIT -PACKARD 
)5«2A 


ur 
<2 

ntacisioN 
DtsTRomrr 
Jta l*i_^2oo 


MAC 


nunoncs 

TCB-1.9P 

Haadptoaa 

Sat 


ipfeoaa  3  J, — T  )  .  oiow  Tuba 

c£:0'.Sj 

- 1  AS  10  T3IQ  | - 

I  ■■  A  I  *  _  J - 1  2 


Povar  lap. 

OT  Drlror 

aad  Log. 

aad  Log. 

Attaauator 

Attaauator 

AS  2D 


I1"  I 


CHORAL 

RADIO 

COMPAJCT 

Oalaaraal 

195*1 


1 


▼SIP  tTRXf 


pACfWT  P4C*IJTjDAC*LPf. 


agate 


Digital  Sua  of  Slaas 
Syatbaalzar  aid 
Saqua aear 


-ft AAAA 


MR 


r  2 


fcCATZ 


R.$MUx 

MCI 


Program  Salaator 
{  juitOM  j 
Coatrol  Loglo 


FIGURE  I  3.  1  :  HARDWARE  MODULES  USED  FOR  STIMULUS 
GENERATION,  DATA  RECORDING  AND  DATA 
PROCESSING.  DOTTED  SECTION  :  THE  RECORDING 
CHAMBER.  SEE  TEXT  FOR  FURTHER  DETAILS. 
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CEP  and  gated-REF  (REF*)  signals  were  recorded  on  an  Analog  Tape  Recorder 
(PRECISION  INSTRUMENTS  MOD:  6200)  set  on  FM  recording  and  1  KHz  cutofT 
frequency,  and  were  available  for  off-line  data  processing.  Since  only  a  4-channel  re¬ 
corder  was  available,  GATE  signals  were  superimposed  (ac-coupled)  on  the  REF  signals 
using  codcr/dccoder  (HM)  modules  generating  the  REF*  signals.  CEP  and  REF  (on¬ 
line)  or  REF*  (off-line)  signals  were  routed  to  the  Lock-In  Amplifier's  (LIA, 
PRINCETON  APPLIED  RESEARCH  EG&G  MOD:  5204)  REF  IN  and  SIG  IN  in¬ 
puts,  respectively.  In  order  to  enhance  the  LIA  input  SNR,  both  CEP  and  REF  or 
REF*  signals  were  passed  through  matched  LPFs  (SKL  MOD:  312)  set  on  an  80  Hz 
comer  frequency. 

An  LIA  is  a  device  that  extracts  the  magnitude  and  phase  of  a  frequency  component 
(coherent  with  a  reference  frequency  applied  to  REF  IN  input)  within  a  noisy  signal 
(applied  to  SIG  IN  input).  The  LIA  technique  is  extremely  sensitive  in  detecting  low- 
amplitude  and  poor  SNR  periodic  signals  (Kaufman  and  Price,  1967;  Regan  and 
Cartwright,  1970;  Euler  and  Kiessling  ,1981;  Nelson  ct  al.,  1984).  Analog  LIA  output 
signals,  proportional  to  the  magnitude  and  phase  of  the  coherent  component,  are  ob¬ 
tained  through  a  single-Time-Constant  (TC)  LPF.  Increasing  the  LIA  output  TC  re¬ 
duces  the  output  signal  variance  and  prolongs  the  response  time  to  a  step  change  in 
MF  (or  modulation  depth). 

The  LIA  equivalent  frequency  response  is  of  a  very  sharply  tuned  Band-Pass  Filter 
(BPF).  Within  the  frequency  range  of  5-61  Hz,  the  LIA  used  in  this  study  (PAR,  MOD: 
5204)  exhibited  average  Equivalent  Noise  Bandwidth  (ENBW)  and  "Q"  factor  values  of 
0.035  Hz  and  530,  respectively  (TABLE  I  3.2).  Note  that  the  stimulus  period  was  always 
made  longer  than  the  LIA  step-function  response-time  (0  to  95%  of  the  full-scale  LIA 
output).  With  an  internally-generated  square  waveform  reference  signal,  only  odd  har- 
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monies,  weighted  by  their  harmonic  numbers,  contributed  to  the  LIA  vector  output. 
These  characteristics  make  the  LIA  a  powerful  tool  in  detection  of  very  low  SNR,  fun¬ 
damental  and  2nd  harmonic  driven  responses  (i.e.  CEP s).  Also,  its  high  frequency  re¬ 
solution  enabled  "locking-in"  on  the  response  from  one  sensory  channel  while  the  other 
sensory  channel  was  excited  with  a  MF  of  slightly  different  value. 


PAR  EG&G  MOD:  5204  LOCK-IN  AMPLIFIER 


TABLE  I  3.2  :  LIA  measured  performance.  TC  :  Time  Constants  of  the  pre-  and 
output-amplifiers,  ENBW  :  Equivalent  Noise  Bandwidth,  "Q"  factor  :  BPF  sharp¬ 
ness,  T  :  response" time,  0  to  95%  of  the  full-scale  LIA  output. 

1  TC  1 ENBW | FREQUENCY  RANGE |"Q"  FACTOR |  T  | 


sec  |  Hz  | 

Hz 

| (Average) 

|  sec 

1  +  1010. 02 1 

1  l 

5  -  15 

|  430 

l 

|  30 

1 

1  1 
1+3  1 0. 05 | 

16  -  61 

i 

|  625 

1 

115 

With  a  CEP  amplification  of  20K-50K,  the  LIA  full  scale  sensitivity  was  set  to 
100-250  millivolt.  An  output  TC  of  1-10  sec  was  selected.  Hard  copies  of  the  LIA 
output  and  GATE  signals  were  obtained  from  a  4-channel  Analog  Chart  Recorder 
(HONEYWELL  MOD:  1400).  A  Spectrum  Analyzer  (HEWLETT-PACKARD  MOD. 
3582A)  was  used  off-line  to  measure  the  EEG  spectrums  (on  64  RMS  averaged  epochs). 

The  system's  overall  magnitude  and  phase  transfer  functions  were  measured  between 
the  stimulus  input  and  the  LIA  output.  The  phase  difference  between  the  REF  fre¬ 
quency  and  the  physical  stimulus  MF  was  found  to  be:  Phase  (degrees)  =  24  -  LOG  10 
(MF),  where  MF  is  the  modulation  frequency  in  Hz.  The  corresponding  magnitude 
transfer  function  displayed  a  maximum  error  of  +/-  0.5  dB  over  the  5-61  Hz  MF  range. 
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All  phase  and  magnitude  results  presented  in  this  study  were  corrected  for  and  should 
be  regarded  as  final,  system-error-free  results. 

In  addition  to  brain  potential  data,  RT  data  were  on-line  measured  and  collected 
using  an  RT  module  (HM).  GATE  signal  transitions  (High-to-Low  or  Low-to-High) 
required  subjects'  responses,  and  turned  on  a  digital  counter.  By  pressing  a  low-pressure 
push-button  switch  (270  gr)  held  in  their  preferred  hand,  the  subjects  stopped  the 
counter.  The  analog-  converted,  digital  number  from  the  counter  was  monitored  on  a 
Multimeter  and  was  recorded  manually. 

3.4  Common  methodology 

3.4.1  Stimulation  procedure 

Subjects  were  seated  in  a  small  closed  room  wearing  a  headphone  set  (TELEPI  ION¬ 
ICS  TDH-49P)  or  optometrist  trial  frames  (AMERICAN  OPTICAL  CORPORATION 
MOD:  11072)  or  both,  holding  a  push-button  switch  in  their  preferred  hand  and  facing 
a  Visual  Stimulator  (HM).  Background  light  intensity  was  130  lux  and  average  noise 
level  did  not  exceed  40  dB  Sound  Pressure  Level  (SPL)  in  the  test  room. 

The  audio  stimuli  (a  915  Hz  carrier,  100%  sinusoidally  Amplitude  Modulated,  AM) 
were  monaurally  delivered  to  the  subject's  left  ear.  The  highest  average  stimulus  inten¬ 
sity  level  (defined  as  "0"  dB  audio  stimulus  level)  delivered  through  the  headphone  set 
was  108  dB  SPL. 

Visual  stimuli  (100%  sinusoidally  AM  white  light)  were  delivered  by  the  Visual 
Stimulator  (VS)  and  binaurally  viewed.  The  VS  was  made  of  a  2.3  cm  diameter,  diffused 
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surface  Glow  tube  light  source  mounted  in  the  center  of  a  61  cm  wide  x  49.5  cm  high 
black  board,  and  viewed  from  a  distance  of  33  cm  (3.6  degrees  of  visual  angle). 


During  visual  stimulation,  subjects  wore  artificial  pupils  (mounted  on  the  trial 
frames)  through  which  they  focused  on  a  small  black  dot  painted  on  the  exposed  Glow 
tube  lens.  Artificial  pupil  diameter  was  1.7  and  both  pupils  were  adjusted  to  allow  an 
unobstructed  view  of  the  Glow  tube  surface,  binaurally  viewed  from  comfortable  head 
posture  (see  FIGURE  I  3.2  for  further  details).  The  highest  average  stimulus  intensity 
level  (defined  as  "0~  dB  visual  stimulus  level)  delivered  by  the  VS  was  44  lux. 

For  both  audio  and  visual  stimuli,  an  MF  waveform  given  by  -COS(2Pi  x  MF  x  t), 
and  a  complete  number  of  M  F  cycles  were  delivered  in  order  to  eliminate  onset/ofiset 
transients.  A  condensed  overview  of  all  the  experiments'  paradigms  is  presented  in  TA¬ 
BLE  1  3.3.  Specific  stimulus  parameters  are  detailed  separately  for  each  experiment. 

3.4.2  Recording  procedures 

Three  types  of  data  were  recorded  from  each  subject:  EEGs  and  CEPs,  RTs  and 
post-experiment  subjective  performance  evaluation.  EEG  and  CEP  data  were  collected 
in  accordance  with  the  research  hypotheses.  The  other  data  were  generated  and  col¬ 
lected  to  promote  control  over  the  experiment  paradigm,  to  verify  response  strategy  and 
induce  motivation  and  involvement.  In  addition  to  task-specific  instructions  and  trials, 
a  general  set  of  instructions  was  stressed  before  each  experiment.  Subjects  were  asked 
to  minimize  their  voluntary  muscle  activity  and  relax,  attend  and  concentrate  on  the 
relevant  stimulus  and  response  cue,  then  react  and  recover  as  quickly  as  possible. 
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THE  VISUAL  STIMULATION  SETUP 


FIGURE  I  3.  2  :  THE  VISUAL  STIMULATOR  AND  THE  ARTIFICIAL  PUPILS 


SPECIFICATIONS  OF  EXPERIMENTS 


TABLE  I  3.3  :  Summary  of  the  experiment  purposes,  stimulus  parameters  and 
recording  procedures.  See  text  for  further  details. 


SPECIFICATIONS  OF  EXPERIMENTS 


TABLE  I  3.3  :  Continued. 


CEPs  were  recorded  for  audio  and/or  visual  stimuli.  Referring  to  the  10/20  electrode 
system  (Jasper,  1958)  auditory  CEPs  were  recorded  between  Cz(  +  IN)-Al(-IN)/Nose 
(driven  reference)  ipsilatcrally  to  the  stimulated  ear,  and  visual  CEPs  were  recorded  be¬ 
tween  Oz(  +  IN)-A1(-IN)/F3  (driven  reference).  EEG  potentials  were  recorded  simul¬ 
taneously  from  both  CEP  recording  sites  to  opened-eyes  condition  without  any  stimulus 
or  required  response.  Skin  impedance  was  kept  below  5K  ohms.  It  was  checked  peri¬ 
odically  and  for  most  of  the  subjects  it  measured  between  1.5K-3K  ohms.  Ag/AgCl 
electrode  leads  (IN  VIVO  METRIC  SYSTEM,  E201)  were  mounted  on  the  skin  (using 
HM  accessories)  in  a  manner  designed  to  ensure  long-term  recording  stability. 

Reacting  to  stimuli  offset  engaged  the  subject's  attention  and  RT  data  was  collected 
to  subjectively  verify  subject  concentration  and  alertness  level  during  the  experiments. 
Subjects  were  required  to  press  a  push-button  switch  (held  in  their  preferred  hand),  as 
quickly  as  possible  in  response  *  •  muli  offset  (additional  onset  responses  were  required 

only  during  the  ATT  expv.r  ..ent).  At  the  end  of  each  run  and  after  submitting  their 
subjective  performance  evaluation,  subjects  were  informed  of  their  actual  scores.  The 
subjective  evaluation  consisted  of  grading  quickness,  attention  and  degree  of  difficulty. 
In  addition,  at  the  end  of  the  AxV  experiment,  subjects  were  asked  to  describe  their  re¬ 
sponse  strategy. 

3.4.3  Data  analysis 

EEG  raw  data  was  RMS  averaged  off-line  (64-128  epochs  were  averaged  per  run), 
and  subjected  to  further  statistical  analysis.  AU  CEP  raw  data  was  recorded  on  the  an¬ 
alog  tape  recorder  and  most  of  it  was  processed  on-line  by  the  L1A.  In  all  the  exper¬ 
iments,  single  Stimulus  duration  (S,  or  S/3  in  AxV  experiment)  and  the  choice  of 
stimulus  repetition  factor  were  the  outcome  of  trade-off  between  performance  and 
practical  considerations.  In  a  pilot  study,  the  TCs  of  the  CEP  responses  to  a  step 
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function  input  stimulus  were  estimated  to  be  less  than  1  sec.  Due  to  the  poor  CEP  SNR, 
LIA  TCs  greater  than  3  sec  had  to  be  applied.  On  the  other  hand,  it  was  desired  to  lower 
the  LIA  TC  in  order  to  minimize  run  duration  and  resource  expenditure  (subject  fees, 
number  of  tapes,  etc.).  Consequently,  LIA  TCs  of  3  and  10  sec  were  used  in  this  study. 

Magnitude  and  phase  values  were  measured  using  the  following  criteria: 

1.  Values  were  accountable  only  for  the  stimulus  "on”  period  of  time,  exceeding  at  least 
2  LIA  output  TCs. 

2.  Magnitude  values  were  recorded  only  if  they  were  accompanied  by  stable  phase  re¬ 
sponses  (CEP  phase  spread  of  less  than  180  degrees). 

Final  values  were  obtained  by  averaging  across  the  repetitions  in  each  experiment  (x2 
or  x3)  for  each  subject.  Linear  averaging  was  applied  for  magnitude  values  and  cyclic 
averaging  (minimizing  the  Standard  Deviation,  SD,  around  the  mean  value)  for  phase 
values.  Data  processing  used  in  specific  experiments  will  be  described  later.  In  light  of 
the  high  inter-subject  variability  observed  in  this  study,  cross-subject  averages  (MEANs) 
were  often  supplemented  with  the  SD  measures  (bars  in  the  figures)  or  the  individual 
data. 

Processed  data  was  statistically  analyzed  and  presented  graphically  using  SAS  and 
SAS/GRAPH  software  packages  (SAS  INSTITUTE,  INC.).  Result  significance  was 
ascertained  by  utilizing  parametric  tests  (t-test,  F-test,  ANOVA,  MANOVA,  regression, 
etc.).  The  original  or  transformed  experiment  data  sets  (i.e.  magnitude  in  dB,  etc.)  were 
all  checked  for  normality.  In  most  cases,  the  normal  data  distribution  assumption  could 
not  be  conclusively  rejected  due  to  border-line  significance  level,  and  a  symmetrical  dis¬ 
tribution  was  observed  (low  absolute  skewness  values).  With  symmetrical  data  distrib¬ 
ution,  non-paramctric  tests,  utilizing  rank  procedures,  testing  the  median  rather  than  the 
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mean,  should  yield  comparable  results  to  parametric  tests.  After  trying  both  tests  on 
several  data  sets,  and  in  light  of  the  general  similarity  found  between  the  results, 
parametric  tests  were  chosen  for  convenience. 
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Part  II:  Unisensory  Continuous  Evoked  Potentials 
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Chapter  4 

Modulation  Transfer  Functions  of  Continuous 


Evoked  Potentials 


4.1  Literature  review 

This  review  will  provide  the  reader  with  background  information  regarding  the  spe¬ 
cific  type  of  Continuous  Evoked  Potentials  (CEPs)  recorded  in  this  study.  A  CEP 
archetype  response  will  be  characterized  in  terms  of  methodological  parameters,  origin, 
peak  magnitudes  and  latencies.  The  visual  CEP  review  was  based  solely  on  comparable 
CEP  studies,  while  the  auditory  CEP  review  was  supplemented  by  additional  information 
from  Transient  Evoked  Potential  (TEP)  studies. 

The  physical  origin  of  scalp-recorded  CEPs  is  generally  deduced  from  their  potential 
distributions  and  neural  delays.  Since  temporal  information  is  not  retained  in  frequency 
domain  methods  (lock-in  technique,  Fourier  analysis,  etc.),  response  delay  can  only  be 
estimated  from  the  CEP  phase  Bode  plot,  under  restricting  assumptions. 
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If  a  linear,  minimum-phase  plus  a  fixed  delay  system  is  assumed,  the  recorded  CEP 
phase  is:  P  =  (2Pi  x  MF  x  D)  +  Po  +  Pf(f).  In  this  equation  MF  is  the  Modulation 
Frequency  in  Hz,  D  is  the  CEP  delay  in  seconds,  Po  is  the  fixed  initial  phase  in  radians 
and  Pf(0  >s  the  frequency-dependent,  neural  filter  phase-shift  in  radians.  The  CEP  delay 
can  be  computed  from:  D  =  d(P)  /  d(MF  x  2Pi)  -  d(Pf(f))  /  d(MF  x  2Pi).  The  first  de¬ 
rivative  term  was  referred  to  as  the  response  "apparent-latency"  (Regan,  1972)  or  simply 
"latency"  in  other  studies.  When  the  second  derivative  term  (the  "group-delay"  of  the 
equivalent  neural  filter)  is  negligible  or  estimable,  the  response  delay  (D)  can  be  ob¬ 
tained. 

The  second  derivative  term  is  negligible  only  when  the  delay  is  estimated  at  frequency 
regions  remote  from  the  poles  and  zeros  of  the  equivalent  filter  modeling  the  peak  CEP 
magnitude.  This  approach,  in  which  the  peak  delay  is  approximated  from  "noisier", 
peak-adjacent  data  (having  lower  magnitudes  and  more  variable  and  unreliable  phases), 
should  not  be  utilized  in  Bode  plots  with  multiple  overlapping  magnitude  peaks  since  the 
approximated  delays  (if  at  all  related  to  their  designated  peaks)  would  consequently  be 
underestimated.  Nevertheless,  this  easy-to-apply  approach  was  used  by  many  research¬ 
ers  (Spekreijse,  1966;  Regan,  1972,  etc.). 

The  second  derivative  term  can  be  estimated  at  the  peak  magnitude  by  fitting  an 
equivalent  linear  model  to  the  CEP  magnitude  response  data.  It  is  possible  to  challenge 
the  validity  of  both  approaches,  but  delay  estimations  of  the  second  approach  (model¬ 
ing)  should  at  the  least  be  more  accurate.  This  is  the  approach  that  has  been  utilized  in 
this  study. 
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4.1.1  Visual  continuous  evoked  potentials  (CEPs) 

4.1. 1.1  CEP  archetype  and  important  stimulation  parameters 

Visual  CEPs,  recorded  in  response  to  amplitude-modulated  light  stimuli,  have  been 
extensively  investigated  and  applied  in  various  fields:  system  identification  studies 
(Twcel  and  Lunel,  1965;  Spekreijse,  1966;  Reits,  1975;  Spekreijse  and  Rcits,  1975; 
Spekrcijse  et  al.,  1977),  normative  and  clinical  studies  (Regan,  1966,  1972,  1977)  and 
human-operator  monitoring  studies  (Wilson  and  O'Donnell,  1981;  Junker  and  Peio, 
1984;  Levison  et  al.,  1985;  Moise,  1985;).  In  these  studies,  changes  in  CEP  parameters 
(magnitude,  phase,- latency,  coherence,  etc.)  were  attributed  to  the  following  exper¬ 
imental  variables: 

1.  Stimulus  intensity  measures  (source  and  background  luminance,  retinal  illumi¬ 
nation),  spatial  pattern,  color,  MF  and  modulation  depth; 

2.  Visual  angle,  stimulus  field,  adaptation,  accommodation  and  pupil  aperture; 

3.  Externally  or  internally  imposed  subject  attitude  (stimulus  relevance,  work-load,  at¬ 
tention  allocation,  etc.). 

The  remainder  of  this  review  will  concentrate  on  a  specific  class  of  CEPs,  namely 
those  recorded  in  response  to  sinusoidally  amplitude  modulated  diffused  (unpatterned) 
light  stimuli.  Generally,  when  response  magnitude  was  plotted  against  MF,  three  major 
peaks  at  10  Hz  (highest),  18  Hz  and  50  IIz  (lowest),  were  typically  found.  These  po¬ 
tentials  were  recorded  under  the  following  general  experimental  conditions: 
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1.  Large  stimulus  field  and  visual  angle  (7-30  degrees,  larger  than  the  fovea  centralis 
field  angle); 

2.  Photopic  stimulation  (luminance  of  20-2000  cd/m2,  retinal  illumination  of  50-5000 
troland)  following  dark  or  stimulus-averaged  intensity  adaptation  (1  troland  is  de¬ 
fined  as  the  retinal  illumination  caused  by  a  I  candela/m2  luminance  surface  viewed 
through  an  exit  pupil  area  of  1  mm2); 

3.  High  stimulus  intensities  and  large  Signal/Noise  Ratio  (SNR)  enhancements  for  the 
high  MF  region  (since  high  modulation  frequency  stimuli  of  30-60  Hz  generated  the 
lowest  CEP  magnitudes); 

4.  Binocular  and  Maxwellian  views  for  low  and  high  stimulus  intensities  respectively; 

5.  Natural  pupil  (natural  focal  depth  control); 

6.  CEPs  recorded  differentially  between  the  occipital  and  the  vertex  area  or  the  earlobe; 

7.  No  attempt  made  to  control  subject  attitude  (attention,  vigilance,  etc.),  except  when 
directly  investigated. 

4. 1.1. 2  CEPs  in  response  to  4-12  Hz  modulation  frequency  stimuli 

CEPs  recorded  for  4-12  Hz  MF  stimuli  showed  a  very  sharp  fundamental  magnitude 
peak  at  the  Alpha  EEG  frequency  (around  10  Hz).  The  potential  distributions  of  these 
CEPs  and  the  Alpha  EEG  spontaneous  activity  were  similar.  It  was  therefore  generally 
accepted  that  these  CEPs,  like  the  Alpha  EEG  spontaneous  potentials,  originated  from 
a  substantial  part  of  the  non-specific  cortical  area  (Spekreijse  et  al.,  1977). 

Within  this  frequency  range,  CEP  latency  estimations  were  difficult  to  assess  due  to 
the  following  obstacles:  (1)  The  wide-spread  CEP  scalp  potential  distribution  could  have 
been  generated  by  multiple,  synchronized  sources.  Consequently,  the  estimated  latency 
of  the  recorded  compound  potential  would  be  affected  by  the  recording  sites.  (2)  The 


degree  to  which  the  spontaneous  Alpha  EEG  and  Alpha  CEP  responses  are  interrelated 
is  still  controversial  (Childers  and  Perry,  1971),  thus  making  the  interpretation  of  the 
CEP  magnitude  and  phase  Bode  plots  unreliable.  In  light  of  the  above,  latencies  of  the 
Alpha  CEP  were  broadly  estimated  and  ranged  between  100-220  msec  (Regan,  1972; 
Diamond,  1977;  Spekreijse  et  al.,  1977;  Junker  and  Peio,  1984).  CEPs  recorded  to  4-7 
Hz  MF  stimuli  (typically  low  magnitude,  highly  non-linear  responses)  were  estimated 
from  the  authors'  figures,  to  have  a  latency  of  220  msec  (Regan,  1966)  and  145  msec 
(Spekreijse,  1966). 

A  two-input  (noise,  stimulus)  one-output  (recorded  cortical  potentials)  sequential 
model  was  derived  by  Spekreijse  (1966)  and  Spekreijse  et  al.  (1977),  based  on  the  fol¬ 
lowing  observations: 

1.  High  ratios  of  2nd  harmonic/fundamcntai  components  that  were  found  in  CEPs  re¬ 
corded  to  4-7  Hz  MFs  (even  for  modulation  depth  as  low  as  1%)  indicated  a  non- 
analytic  non-linear  processing  stage.  Based  on  further  analysis,  transfer  function  of 
this  stage  was  modeled  by  an  asymmetric  full-wave  rectifier. 

2.  The  fundamental  and  2nd  harmonic  CEP  components  at  the  Alpha  EEG  MF  region 
had  similar  Bode-plots  and  scalp  potential  distributions.  This  behavior  was  modeled 
by  a  cortical  equivalent  BPF,  sharply  tuned  at  the  Alpha  EEG  frequency  and  sub¬ 
jected  to  visual  and  cerebral  noise  input. 

3.  Saturation  of  CEP  magnitudes,  induced  by  an  increase  in  modulation  depth,  was 
found  to  be  MF-dependent  and  binocularly  additive.  This  analytic  non-linear 
processing  stage  was  modeled  by  a  "soft’-saturation  element,  limiting  the 
binocularly-merged  rectifier  stage  output. 

4.  Superimposed  auxiliary  signals  (noise  or  other  sinusoidally  Amplitude  Modulated, 
AM,  signal)  linearized  the  responses  evoked  for  stimuli  with  MF  of  Alpha  EEG 
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frequency/2  (increasing  the  fundamcntal/2nd  harmonic  component  ratio).  The  de¬ 
pendency  of  the  linearizing  effect  on  the  auxiliary  signal  frequency  indicated  a  linear 
processing  stage  preceding  the  non-linear  rectifier  stage.  This  linear  stage  was 
modeled  by  a  retinal  equivalent  BPF  with  a  center  frequency  of  18  Hz  and  low  and 
high  frequency  asymptotes  of  -6  and  -18  dB/octave,  respectively. 

This  working-model  (Spekreijse,  1966)  was  formulated  under  the  following  assump¬ 
tions:  retina-cortex  sequential  information  processing  (no  feedback,  no  parallel  path¬ 
ways,  etc.),  single  output  system  (the  recorded  potentials  originated  from  a  cortical  single 
source),  time-invariant  sensory  channel,  zero-memory  non-linear  elements  and  non¬ 
interactive  sensory  pathway  (unaffected  by  cross-sensory  input,  subject  attitude,  etc.). 
Some  of  these  assumptions  will  be  contested  later  in  the  discussion  of  the  MTF  exper¬ 
iment. 

4. 1.1.3  CEPs  in  response  to  12-25  Hz  modulation  frequency  stimuli 

CEP  magnitudes  at  the  12-25  Hz  MF  region  were  maximal  at  the  occipital,  4  cm  right 
or  left  ofTthe  midlinc,  ipsilateral  to  the  stimulated  visual  field.  It  has  not  yet  been  re¬ 
solved  whether  these  CEPs  were  actually  generated  in,  or  just  favorably  recorded  from, 
the  ipsilateral  hemisphere.  Regan  (1972)  and  Spekreijse  et  al.  (1977)  concluded  that  the 
equivalent  dipole  source  of  these  CEPs  resided  in  the  secondary  visual  cortex  (Brodmann 
areas  18  and  19;  the  "secondary  response").  The  estimated  latency  range  of  these  CEPs 
was  60-140  msec  (Regan,  1966;  Regan,  1972;  Diamond,  1977;  Spekreijse  et  al.,  1977, 
Junker,  1984). 

The  "secondary  response"  CEPs  were  characterized  by  the  following  observations: 
the  responses  were  color-dependent  and  relatively  undistorted  (high 
fundamental/harmonic  component  ratios);  the  spontaneous  EEG  spectrum  did  not  show 
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any  12-25  Hz  frequency  band  equivalent  selectivity  (the  response  could  not  be  modeled 
by  a  cortical  equivalent  filter  subjected  to  cerebral  noise).  The  distal,  linear  transfer 
function,  of  the  "secondary  response"  was  modeled  by  a  linear  equivalent  BPF  with  a 
center  frequency  of  18  Hz  and  low  and  high  frequency  asymptotes  of  *6  and  -12  to  -18 
dB/octave,  respectively  (Spekreijse  et  al.,  1977). 

4.1. 1.4  CEPs  in  response  to  45-60  Hz  modulation  frequency  stimuli 

CEP  magnitudes  at  the  45-60  Hz  MF  region  were  maximal  around  the  occipital  re¬ 
gion,  topographically  corresponding  to  the  projected  retinal  map  of  the  stimulated  fovea 
centralis  (Spekreijse,  1966;  Spekreijse  et  al.,  1977).  The  primary  visual  cortex  (Brodmann 
17)  was  suggested  as  the  equivalent  dipole  source  for  these  low  level  CEPs  (the  "primary 
response").  Estimated  latency  range  for  these  CEPs  was  30-60  msec  (Tweel  and  Lunel, 
1965,  for  MFs  >  35  Hz;  Regan,  1972;  Spekreijse  et  al,  1977). 

4.1.2  Auditory  continuous  evoked  potentials 

Contrary  to  the  visual  system,  few  comparable  CEP  studies  have  been  carried  out  on 
the  human  auditory  system.  Utilizing  amplitude  modulated  tone  or  noise  and  click 
stimuli,  CEPs  recorded  for  4-15  Hz,  15-25  Hz  and  40-55  Hz  MF  stimuli  were  associated 
with  latency  ranges  of  60-200  msec  (Rodenburg  et  al,  1972;  Rickards  and  Clark,  1984), 
30-100  msec  (Rees,  1981;  Rickard  and  dark,  1984;  Stapells  et  al,  1984)  and  10-50  msec 
(Galambos  et  al.,  1981;  Gaiambos,  1982;  Rees,  1981;  Rickards  and  Clark,  1984;  Stapells 
et  al.,  1984),  respectively.  Some  of  the  latency  measures  cited  here  were  not  provided 
by  the  above  authors,  but  were  estimated  from  their  phase  plot  data.  A  common  feature 
in  all  the  cited  studies  is  that  subject  attitude  was  not  formally  controlled.  The  subjects 
were  reading  a  book,  relaxing  or  dozing. 
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Rickards  and  Clark  (1984)  obtained  averaged  CEPs  (recorded  from  the  vertex- 
mastoid)  for  0.25-8  KHz  tones,  94%  modulation  depth,  4-512  Hz  sinusoidally 
amplitude-modulated  stimuli,  at  average  sound  levels  of  20-108  dB  SPL.  They  found 
that  the  fundamental  magnitude,  generally  the  dominant  CEP  component,  was  carrier- 
frequency-dependcnt.  Maximal  magnitudes  at  low  and  high  MFs  (relative  to  20  Hz) 
were  obtained  with  low  and  high  carrier  frequencies  (relative  to  1  KHz),  respectively. 
Bode  plots  of  CEPs  recorded  from  one  of  their  subjects  showed  two  similar  magnitude 
peaks  at  15  and  40  Hz,  superimposed  on  an  LPF-like  response  (60  Hz  corner  frequency 
and  -8dB/octave  high  frequency  attenuation).  Estimated  latencies  of  the  CEPs  recorded 
for  the  4-12  Hz,  15-35  Hz  and  40-140  Hz  MF  ranges  were  150,  50  and  15  msec,  respec¬ 
tively. 

Rees  (1981)  obtained  averaged  CEPs  (recorded  from  F4-A2),  for  a  1  KHz  carrier 
tone,  100%  modulation  depth,  1-500  Hz  sinusoidally  amplitude-modulated  stimuli,  at 
intensities  up  to  70  dB  Sound  Level  (SL).  Magnitude  peaks  were  found  at  4  Hz  (major 
peak,  in  one  subject)  and  40  Hz  (minor  peak,  in  two  subjects).  Average  estimated 
latencies  of  the  CEPs  recorded  at  the  5-5.5  Hz,  18-21  Hz  and  40-55  Hz  MFs  were  300, 
1 10  and  50  msec,  respectively. 

Rodenburg  et  ai.  (1972)  obtained  averaged  CEPs  (recorded  from  C3-F3.4)  for  white 
noise  sound,  5-100%  modulation  depth,  4-11  Hz  sinusoidally  amplitude-modulated 
stimuli,  at  an  intensity  level  of  55  dB  SL.  They  demonstrated  (in  two  subjects)  a  single 
fundamental  magnitude  peak  (the  dominant  harmonic  in  their  recorded  CEPs)  at  the 
8-10  Hz  MF  range.  Neither  of  the  previously  cited  studies  (Rees,  1981;  Rickards  and 
Clark,  1984)  revealed  a  similar  peak.  Galambos  et  al.  (1981)  obtained  averaged  CEPs 
(recorded  from  the  forehead-earlobe)  for  10-60/sec,  click  or  tone  burst  stimuli,  at  inten¬ 
sity  levels  up  to  50  dB  SL.  They  plotted  response  amplitude  against  stimulus  rate  and 
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found  two  magnitude  peaks  at  stimulus  rates  of  15-20  (minor)  and  35-45  (major) 
stimuli/scc.  It  was  not  clear  how  the  amplitudes  of  the  transient  evoked  responses  were 
defined  and  measured.  This  kind  of  ambiguity  can  be  resolved  by  using  a  Lock-In  Am¬ 
plifier  (LI A),  working  in  the  frequency  domain. 

4.2  Methodology 

4.2.1  Data  acquisition  paradigm 

The  investigated  MF  range  of  5-61  Hz  was  divided  into  four  sections  (runs)  presented 
in  the  following  order:  5-8.5  Hz  (8  MF  values,  12.5'  run  duration  in  minutes),  16.1-27.7 
Hz  (9  MF  values,  8'  run  duration).  9-15.3  Hz  (13  MF  values,  19'  run  duration)  and 
30.5-61  Hz  (6  MF  values,  9'  run  duration).  The  combinations  of  stimulus  duration-LIA 
output  Time-Constant  (TC)  setting  were  a  function  of  VF  :  30-51  sec-10  sec,  16-28 
sec-3  sec,  29-50  sec- 10  sec  and  18-36  sec-3  sec  for  the  first,  second,  third  and  fourth 
runs,  respectively.  Within  a  run,  MF  values  were  non-monotonically  stepped  (1st,  mid¬ 
dle  (m),  2nd,  m+  1,  3rd,  m  +  2,..  etc.)  to  reduce  possible  temporal  interaction  between 
adjacent  MF  values.  This  type  of  interaction  was  demonstrated  and  reviewed  in  Kay 
(1982)  and  others  for  the  auditory  system.  Inter  Stimulus  Interval  (ISI)  was  set  at  3.2 
sec  and  every  MF  value  was  repeated  twice  (or  three  times  if  it  was  the  first  MF  of  the 
run).  The  entire  "MTF"  and  "EEG"  recording  session,  auditory  first  and  visual  second, 
lasted  four  hours,  including  rest  breaks  taken  between  runs  (more  details  in  TABLE  I 
3.3). 

In  order  to  maximize  the  CEP  magnitudes,  high  stimulus  intensities  were  utilized  (108 
dB  SPL  or  730  cd/ml  luminance  level,  defined  as  "0"  dB).  Modulation  Transfer  Func¬ 
tions  (MTFs)  of  cross-rcpetition-averaged  magnitude  and  continuous-phase  values 
plotted  against  MF,  were  generated  for  each  subject.  Following  the  CEP  session, 
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spontaneous  EEG  was  recorded  simultaneously  from  the  same  two  recording  sites  under 
opened-eyes,  no-stimulus  and  no-required-response  conditions.  EEG  spectrums  were 
estimated  from  RMS-avcragcd  spontaneous  EEG  activity 

4.2.2  Data  analysis 

4. 2. 2.1  Latency  estimation 

Assuming  a  linear,  minimum-phase  plus  a  fixed  delay  system,  CEP  latencies  (and 
consequently  latency  MTFs)  were  estimated  for  each  subject  from  the  continuous-phase 
MTF  slope.  Regression  and  correlation  coefficients  were  computed  on  a  5  phase- 
point-wide  sliding  window.  CEP  latencies  (regression  coefficients,' 2Pi)  and  their  reli¬ 
abilities  (correlation  coefficients)  were  assigned  to  the  windows'  center  frequencies.  The 
chosen  sampling  window  width  enabled  sensitive  detection  of  latency  variations  with 
reasonable  regression  coefficient  significance  levels.  Note  that  shorter  latencies  obtained 
for  higher  MFs  will  be  more  affected  by  a  fixed  margin  error  phase  plot,  imposed  by  the 
recording  system  and  measurement  process.  Therefore,  demonstrating  the  dissimilarity 
of  shorter  latency  populations  is  difficult  using  this  data  analysis  technique. 

For  latency  estimations,  the  L1A  cyclic-phase  output  data  (-180  degrees  to  +180 
degrees)  had  to  be  unfolded  into  continuous-phase  MTFs.  Although  an  important 
stage,  phase  unfolding  has  been  usually  conducted  informally  and  intuitively.  A  more 
objective  procedure  was  used  by  Levison  et  al.  (1985).  Their  phase  data  was  fitted  to  a 
linear  filter  model,  estimated  from  the  magnitude  MTF.  The  underlying  assumptions 
were  of  a  minimum-phase,  sequential  processing  system  and  high  inter-/intra-subject 
variability  ratio.  A  more  heuristic,  less  restrictive  approach  was  applied  in  the  current 
study  (FIGURE  II  4.1).  The  unfolded,  cross-subject-averagcd  cyclic-phase  plot  was 
used  as  a  "template"  against  which  the  individual  cyclic-phase  plots  were  matched  and 
unfolded.  Complementary  and  redundant  data  (from  other  experiments  in  this  study) 
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helped  to  determine  the  phase  values  of  the  noisier  measurements  (applied  to  less  than 
5%  of  the  data). 

4.2.2.2  Neural  delay  estimations 

Neural  delays  were  estimated  for  the  auditory  and  visual,  Alpha,  Beta  and  Theta 
frequency  region  magnitude  peaks.  Linear,  minimum-phase  plus  a  fixed  delay  neural 
system  was  assumed,  and  the  CEP  magnitude  distributions  were  modeled  by  two  conju¬ 
gate  poles,  LPF  or  BPF,  for  simplicity. 

In  order  to  increase  their  reliability,  models  were  fitted  to  preprocessed  magnitude 
data  sets: 

1.  Theta,  Alpha,  Beta  and  *40  Hz"  MF  regions  were  rejected  from  the  magnitude  data 
set  if  no  distinct  magnitude  peak  was  visually  recognized  within  the  5-8  Hz,  8-12 
Hz,  14-28  Hz  and  32-61  Hz  frequency  ranges,  respectively. 

2.  Each  regional  magnitude  MTF  was  expressed  in  dB  gains  relative  to  its  peak  mag¬ 
nitude  value  and  plotted  against  frequency  ratios  relative  to  its  peak  frequency 
value.  This  alignment  procedure  was  employed  in  order  to  obtain  a  more  accurate 
description  of  the  neural  equivalent  filter  model  (reducing  the  inter-subject  variabil¬ 
ity). 

3.  In  each  of  the  signal  x  MF  region  groups,  only  the  data  of  magnitude  values  ex¬ 
ceeding  some  threshold  level  were  modeled.  The  threshold  level  was  arbitrarily  de¬ 
fined  as  the  lower  quartile  value  of  the  data  distribution.  This  procedure  insured 
against  the  inclusion  of  noisier,  less  reliable  data  values  in  the  analysis. 
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PHASE  UNFOLDING  AND  LATENCY  DERIVATION  PROCEDURES 
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FIGURE  II  4.  1  :  STEPS  TAKEN  IN  GENERATING  THE  PHASE  AND 
LATENCY  MODULATION  TRANSFER  FUNCTIONS 
(MTFs).  THE  CYCLIC  PHASE  (Pcyc)  WAS 
UNFOLDED  INDEPENDENTLY  OF  THE  MAGNITUDE  MTF, 
GENERATING  THE  CONTINUOUS  PHASE  (Peon). 

SD  :  STANDARD  DEVIATION. 


Filter  parameters  (LPF  or  BPF  and  damping  coefficients)  were  optimized  in  the 
Mean  Square  Error  sense  (MSE,  errors  being  the  model  -  data  gain  differences).  As 
previously  described  (in  section  II  4.1),  it  is  possible  to  estimate  the  neural  delay  from 
the  difference  between  the  data  and  the  model  phase  slopes  measured  at  the  peak  mag¬ 
nitude.  The  slopes  were  derived  from  regression  coefficients  calculated  for  the  frequency 
range  of  1/1.15  to  1.15  of  the  frequency  region's  center  frequency  (0.85  to  1.17  range 
was  used  for  the  "40  Hz"  MF  region).  Center  frequencies  were  obtained  by  averaging 
the  individual  peak  magnitude  frequencies. 

4.3  Results 

4.3.1  Spontaneous  EEG  spectrums 

Cross-subject  averages  of  spontaneous  EEG  spectrums  recorded  from  Oz-Al  and 
Cz-Al  sites  (FIGURE  II  4.2)  were  significantly  different  only  at  frequencies  below  10 
Hz  (Alpha  EEG),  where  larger  vertex  potentials  were  recorded.  Both  spectrums  arc 
characterized  by  a  similar,  inter-subject  highly-variable  peak  at  Alpha  EEG  frequency, 
absence  of  additional  higher  frequency  peaks  and  similar  Coefficient  of  Variability  (de¬ 
fined  as:  CV  =  MEAN  /  Standard  Deviation,  SD). 

4.3.2  Lock-in  amplifier  raw  data  output  example 

One  of  the  better  CEP  sets  processed  by  the  LI  A,  is  demonstrated  in  FIGURE  II  4.3. 
This  data  was  recorded  in  four  runs  covering  the  5-61  Hz  MF  range.  The  data  of  each 
run  details  the  MFs,  stimulus  durations  and  LI  A  fundamental  component  vector  output. 
These  LIA  outputs  were  used  to  generate  the  magnitude,  phase  and  latency  MTFs  pre¬ 
sented  in  the  following  sections. 
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SPONTANEOUS  EEG  MAGNITUDES 
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FREQUENCY  IN  HZ 

FIGURE  □  4.2  :  EEG  SPECTRUM3,  AVERAGED  (MIAN  AND  SD)  OVER  9  SUBJECTS. 

STAR :  RECORDED  FROM  CZ-Al/NOSE  (AUDITORY  CEP)  SITES, 
DIAMOND  :  RECORDED  FROM  02-A1/F3  (VISUAL  CEP)  SITES, 
HORIZONTAL  BARS  :  SD  OF  THE  MEAN  VALUES. 
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LOCK-IN  AMPLIFIER  (LIA)  OUTPUT  EXAMPLE 


FIGURE  II  4.3  :  LIA  OUTPUT  OF  4  CONSECUTIVE  RUNS  OF  VISUAL  CEP 
RAV  DATA  OBTAINED  FROM  SUBJECT  SC  UNDER  "0"  dB 
INTENSITY  LEVEL  AND  LIA  O' fTPUT  TIME  CONSTANTS 
(TCs )  OF  3  AND  10  SECONDS.  EACH  RUN  CONTAINS 
MAGNITUDE  AND  PHASE  CEPs ,  PHASE  POLARITY  AND 
MODULATION  FREQUENCIES  (MFs)  AND  THEIR  GATES. 
EACH  MAJOR  DIVISION  EQUALS  30  SECONDS. 


4.3.3  CEP  magnitude  modulation  transfer  functions 

Auditory  and  visual  cross-subject  averages  of  magnitude  MTFs  (FIGURE  II  4.4) 
have  a  similar  general  shape,  different  from  the  corresponding  spontaneous  FFG  spcc- 
trums.  Both  averaged  MTFs  exhibit  a  primary  peak  (1. 5-2.5  microvolt  RMS)  at  5-6  Hz 
and  a  secondary  peak  (0.5-1  microvolt  RMS)  at  16-25  Hz  (Theta  and  Beta  MF  regions, 
respectively).  The  lower-magnitude  auditory  MTF,  shows  additional  peaks  (0.6-0. 9 
microvolt  RMS)  at  10.5  Hz  and  50  Hz  (Alpha  and  ”40  Hz"MF  regions,  respectively). 
In  general,  the  auditory  MTF  demonstrates  smaller  absolute  (SD)  and  relative  (SD  / 
MEAN)  inter-subject  variabilities. 

When  t-tested  for  Beta  and  Theta  MF  regions,  auditory  and  visual  averaged  MTFs 
significantly  differed  in  absolute  values,  but  were  similar  in  relative  values  (magnitude 
ratio  of  Theta/ Beta;  TABLE  II  4.1).  A  further  full-frequency  scale  comparison  of  mag¬ 
nitude  dynamic  range  (95%-5%  of  the  magnitude  range)  and  magnitude  median  (ME¬ 
DIAN)  revealed  an  obvious  trend  of  larger  visual  averaged  CEPs.  This  trend  was  not 
significant  for  the  means  (P  >  |t|  of  0.093-0. 14),  although  the  distributions  were  found 
to  be  significantly  different  (P  >  F  of  0.009-0.0004). 

4.3.4  CEP  phase  modulation  transfer  functions 

Demonstrations  of  the  previously  described  phase  processing  stages  (FIGURE  II 
4.1),  "template"  generation  -->  alignment  procedure  —  >  "match  goodness"  test,  arc 
presented  in  FIGURES  II  4.5,  4.6  and  4.7,  respectively.  Auditory  and  visual  cross¬ 
subject  averages  of  the  continuous-phase  MTFs  are  shown  in  FIGURE  II  4.8.  The 
overall  visual  phase  response  dynamic  range  (6Pi  radians)  was  significantly  smaller,  due 
to  a  significant  auditory  -  visual  high-frequency  phase  difference  (TABLE  II  4.1).  Other 
inter- sensory  significant  differences  were  found  at  Alpha  and  Beta  MF  regions  (P  >  |t| 
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of  <  0.0001).  Both  average  MTFs  exhibited  small  variance  at  Beta  and  Theta  frequency 
regions,  and  larger  variance  at  Alpha  MF  region. 

4.3.5  CEP  latency  modulation  transfer  functions 

Auditory  and  visual  cross-subject  averages  of  the  latency  MTFs  and  their  confidence 
levels  arc  presented  in  FIGURE  II  4.9  and  compared  in  TABLE  II  4.1.  Generally, 
longer  latencies  were  associated  with  responses  elicited  for  lower  MFs  and  vice  versa. 
Cross-modality  averages  of  latency  MTFs  at  the  Theta,  Alpha,  Beta  and  "40  Hz"  MF 
regions  were  220  msec,  160  msec,  70  msec  and  25  msec,  respectively  (FIGURE  II  4.9, 
TABLE  II  4.1).  Auditory  and  visual  MTFs  exhibited  similar  latency  MTFs  at  Beta  and 
higher  MF  regions. 

4.3.6  CEP  modeling  and  neural  delays 

Linear  models  of  the  pre-processed  auditory  and  visual  CEPs  (see  section  II  4. 2.2.2 
for  details)  at  Theta,  Alpha,  Beta  and  "40  Hz"  MF  regions  are  presented  in  FIGURE 
II  4.10  and  described  in  TABLE  II  4.2.  A  "40  Hz"  MF  region  model  for  the  visual  CEPs 
was  not  constructed  since  only  two  subjects  exhibited  magnitude  MTF  peaks  at  MF 
above  32  Hz.  A  LPF  model  was  justified  (MSE-wise)  only  for  the  auditory  Alpha  MF 
region.  In  general,  the  auditor}'  data  were  fitted  with  sharper  filters  (higher  Q  factor), 
and  the  Alpha  MF  region  filters  in  both  modalities  were  the  sharpest.  Also,  longer  and 
more  reliable  delays  were  estimated  for  the  visual  CEPs,  where  longest  and  shortest  de¬ 
lays  were  found  in  both  modalities  at  Alpha  and  Beta  MF  regions,  respectively.  Cross¬ 
modality  averages  of  neural  delays  were  59  msec,  82  msec,  31  msec  and  10  msec  for  the 
Theta,  Alpha,  Beta  and  "40  Hz"  (auditory  CEP  only)  MF  regions,  respectively. 
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FIGURE  □  4.4  :  MAGNITUDE  RESPONSES,  AVERAGED  (MEAN  AND  SD)  OVER  9  SUBJE 
DOTTED  LINE  :  NOISE  LEVEL  IN  MICROVOLT  RMS, 

STAR  :  AUDITORY  CEP  (RECORDED  PROM  CZ-A1/NOSE  SITES), 
DIAMOND  :  VISUAL  CEP  (RECORDED  FROM  0Z-A1/F3  SITES), 
HORIZONTAL  BARS  :  SD  OF  THE  MEAN  VALUES. 
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MODULATION  FNCOUCNCT  IN  HZ 

FIGURE  n  4.5  :  avquce  phase  rebpchse  over  b  SUBJECTS,  STAS :  AVERAGE  (8D  Knaaziiic 
MEAN}  CTCllC  PHASE,  VDntCALUME  :  STANDARD  ECVIABOM  SPREAD. 

VISUAL  INDIVIDUALS  AND  AVERAGED  CYCLIC -TO -CONTINUOUS  CONVERTED  PHASES 


FIGURE  n  4.8  :  PHASE  RESPONSE  or  I  SUBJECT*,  Lira :  AVStACD  CTOJC,  COHTntM)  to  ooirmroous 
phase,  chicles :  wnrouAiA  CTaic-7P-cam*uoua  coothhep  phase. 

VISUAL  CONTINUOUS  AND  CYCUC -TO -CONTINUOUS  CONVERTED  AVERAGE  PHASES 


FIGURE  □  4.7  :  AVERAGE  PHASE  KiHPUKBZ  OVER  «  SUBJECT*,  STAS :  AVERAGE)  CTOJC,  ClMfUUBA  TO 
COimSUOUB  PHASE,  DOAKU  :  AVERAGED,  CYCLIC  CUNIU1B  TO  OOSTOIDOUS  PHASE. 
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CEP  CONTINUOUS  PHASES 
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CEP  LATENCIES 


FIGURE  n  4.9  :  AVERAGED  CONFIDENCE  (TOP)  AND  LATENCY  (BOY)  OVER  9  SUBJECTS. 

LATENCY  (REGRESSION  COEIT/fl.89)  AND  CONFIDENCE  (CORRELATION 
COOt)  WERE  ESTIMATED  FROM  S  PHASE-POINT-WIDE  SLIDING  WINDOW. 
STAR :  AUDITORY  CEP  (CZ-Al/NOSE  RECORDING  STIES), 

DIAMOND  :  VISUAL  CEP  (OZ-Al/PS  RECORDING  SITES), 

HORIZONTAL  BARS  :  SD  OF  THE  MEAN  VALUES. 
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AUDITORY  AND  VISUAL  CEP  COMPARISON 


TABLE  II  4.1  :  t-test,  F-test  and  descriptive  statistics.  MEAN  :  averaged  value  over 
N  observations,  SD  :  Standard  Deviation,  MF  :  Modulation  Frequency.  MTF: 
Modulation  Transfer  Function,  BETA:  18,  19.1,  20.3  and  21.8  Hz  MF  values, 
THETA  :  5,  5.5  and  6  Hz  MF  values. 


PARAMETERS 

IN 

AUDITORY 

|  VISUAL 

1  P>F  | P> | t | | 

MEANiSD  of 
Magnitudes  in 
microvolt  RMS 

1 

1 

1 

1 

1 

i  i  i 

1  1  1 

1  1  1 

THETA  MF  region 

1 27 

1 

1. 4110. 53 

1 

|  2.1611.57 

i 

1  1  1 

1 0. 000 1 0.  026 | 

i  l  i 

BETA  MF  region 

1 36 

l 

0. 6410. 23 

1 

|  0.9010.46 

i 

1  1  1 

|0. 000 10.  004 1 

i  i  i 

THETA  /  BETA 

i 

1 9 

I 

2.  4211.  10 

1 

|  2.4711.48 

i 

1  1  ! 

|  0.42|  0.  94 1 

MTF  MEDIAN 

1 

1 9 

l 

0. 60±0. 13 

1 

|  0. 8310.  35 

i 

1  1  1 
10.0091  0.09  1 

MTF  95%-5%  range 

1 

1 9 

1.  43±0.  34 

1 

|  2.2411.47 

1  1  1 
|0.000|  0.  14  I 

MEAN±SD  of  con- 

1 

1 

i  i  i 

tinuous  phases  ini 

1 

i  i  i 

radians 

1 

i  i  i 

THETA  MF  region 

1 27 

-6.  07±1. 1 

1 

|  -5.9811.3 

l 

i  i  i 

|  0.  331.0.  79  1 

1  1  1 

BETA  MF  region 

|  36 

-18.  22±1. 05 

1 

|-16. 8310.  97 

i 

I  1  1 

|  0. 65 | 0. 000 1 

THETA  -  BETA 

|  9 

12.  15±0. 85 

1 

|  10.8511.81 

i 

1  1  1 
10.047]  0.07) 

MTF  95%  value 

1 9 

-5. 97±0. 69 

1 

|  -5.8211.38 

I 

1  1  1 
|  0.07|  0.  78( 

MTF  5%  value 

1 9 

-26. 22±1. 55 

1 

|  -24.  4111. 4 

i 

!  !  1 
|  0.  79  1 0.  019( 

MTF  95%-5%  range 

1 9 

20.  95±1.  5 

1 

|  18.5911.63 

1  1  1 
|  0.  82 1 0.  038  | 

MEANiSD  of 
latencies  in  msec 

1 

! 

i  i  i 

i  i  i 

THETA  MF  region 

|  9 

2061143 

1 

|  2331145 

i  i  i 

|  0. 961  0.  701 

(at  6  Hz  ) 

1 

1  1  1 

BETA  MF  region 

|  36 

69132 

!  67128 

|  0. 40  j  0. 73  j 

(over  all) 

1  1  1 
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UNEAR  MODELS  OF  CEP  RESPONSES 

S!G=AUDI0  REGION=THETA  CF=5.45  HZ  THRESH»0.70  MICROVOLT  RMS 


FREQUENCY  RATIO  (LOG  SCALE?. 

FIGURE  n  4.10  :  UREAS  MODELING  AT  ALPHA,  BETA  AND  THETA  ALIGNED  ME  SSQONB. 

LEAST-M8E  QUADRATIC  BPf  MOOEUi  WERE  RITCD  TO  THE  MAGNITUDE  DATA. 

CT  CENTER  FREQUENCY,  THRESH:  DATA  DISTRIBUTION  LOWES  QUARTOS. 

URCEB  CHARACTERS:  DATA  ECCEEDDK!  THE  THRESHOLD  LEVEL  (MODSLEC  DATA). 
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LINEAR  MODELS  OF  CEP  RESPONSES 

SIG-AUDiO  REGION-ALPHA  CF-10.35  HZ  THRESH-0.40  MICROVOLT  RMS 


FREQUENCY  RATIO  CLOG  SCALE) 

FIGURE  114.10  :  LINEAR  MODELING  AT  ALPHA,  BETA  AND  THETA  ALIGNED  UP  RBGK1N8. 

LEAST-MS  QUADRATIC  HPT  MODELS  VISE  FTTTH)  TO  TOE  MAGNITUDE  DATA. 

CFi  CENTER  IMQUENCT,  THRESH;  DATA  DISTRIBUTION  LORES  QUARTOS, 

LARGER  CHARACTERS:  DATA  ESCBDDIO  TOE  THRESHOLD  LEVtt  (MODELED  DATA). 
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0.58  0.66  0.76  0.87  1.0  1.15  1.32  1.51  1.74 

FREQUENCY  RATIO  (LOG  SCALE) 

FIGURE  n  4.10  :  LINEAR  MODEIiNG  AT  ALPHA,  BETA  AND  THETA  AUONED  MF  REGIONS. 

LEAST-USE  QUADRATIC  BPP  MODELS  WERE  FITTED  TO  THE  MAGNITUDE  DATA. 

CT:  CENTER  FREQUENCY.  THRESH;  DATA  DISTRIBUTION  LOVER  QUARITLE, 

LARGER  CHARACTERS:  DATA  EXCEEDING  THE  THRESHOLD  LEVEL  (MODELED  DATA). 
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0.5B  0.66  0.76  0.B7  1.0  1.15  1.32  1.51  1.74 

FREQUENCY  RATIO  (LOG  SCALE) 


nova  D  4.10 :  UNEAB  UODBiNC  AT  BKSB  PUQUBVCY  ("40  HZ")  ALIGNED  MF  REGION. 

LEAST-MSS  QUADRATIC  BP7  MODEL  WAS  FITTED  TO  THB  MAGNITUDE  DATA. 

CFi  CERTtH  rMQUBWT.  THRESH;  DATA  DMTBgUTlOlf  LOTBt  QUARTOS. 

larger  characters;  data  ncnm  res  threshold  levs,  (modeled  data). 
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LINEAR  MODELS  OF  CEP  RESPONSES 

S I G=V I SUAL  REGION=THETA  CF=5.40  HZ  THRESH-0.70  MICROVOLT  RMS 
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FIGURE  H  4.10  :  UNEAR  MODELING  AT  ALPHA.  BETA  AND  THETA  ALIGNED  MF  REGIONS. 

LEAST-USE  QUADRATIC  HP?  MODELS  WERE  FITTED  TO  TEE  MAGNITUDE  DATA. 

CF-.  CENTER  FREQUENCY,  THRESH:  DATA  DISTRIBUTION  LOWER  QUARTS*, 

LARGER  CHARACTERS:  DATA  EXCEEDING  THE  THRESHOLD  LEVEL  (MODELED  DATA). 
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0.5B  0.66  0.76  0.B7  1.0  1.15  1.32  1.51  1.74 

FREQUENCY  RATIO  (LOG  SCALE) 

nGUHEII4.10:  UNEAR  MODELING  AT  ALPHA.  BETA  AND  THETA  ALIGNED  Iff  REGIONS. 

LEAST-USE  QUADRATIC  HPT  MODELS  WERE  FOTH)  TO  THE  MAGNITUDE  DATA. 

CT:  CENTER  FREQUENCY,  THRESH:  DATA  DISTRIBUTION  LORHi  QUANTILE, 

LARGER  CHARACTERS:  DATA  EXCBUINC  THE  THRESHOLD  LEVS.  (MODELED  DATA). 
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0.58  0.66  0.76  0.87  1.0  1.15  1.32  1.51  1.74 

FREQUENCY  RATIO  (LOG  SCALE) 


FIGURE  n  4.10  :  LINEAR  MODELING  AT  ALPHA,  BETA  AND  THETA  ALIGNED  MF  REGIONS. 

LEAST-USE  QUADRATIC  HPF  MODELS  FERE  FITTED  TO  THE  MAGNITUDE  DATA. 

CF:  CENTER  FREQUENCY.  THRESH;  DATA  DISTRIBUTION  LOWER  QUAKTILE, 

LARGER  CHARACTERS:  DATA  EXCEEDING  THE  THRESHOLD  LEVEL  (MODELED  DATA). 
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ESTIMATION  OF  CEP  DELAYS  AND  LATENCIES 


TABLE  II  4.2  :  Estimated  CEP  delays  and  latencies  from  band-pass  or  low-pass 
quadratic  filter  models.  Max  group  delay  :  filter's  largest  phase-shift  slop,  SDSE  : 
Standard  Deviation/Standard  Error,  MF:  Modulation  Frequency,  RMSE:  Root 
Mean  Square  Error. 


AUDITORY  CEP  MF  REGIONS 


PARAMETERS  j 

THETA  | 

ALPHA  | 

BETA  | 

"40  Hz" 

Frequency  Range,  Hz  j 

5-8  | 

8-12  | 

14-28  | 

32-61 

Number  of  Cases  (N)  j 

8  1 

5  | 

8  I 

8 

Peak  FrequencyiSD ,  Hz  j 

5.  4±0.  7  | 

10.410.8  | 

18.911.2  | 

4514. 4 

Peak  Mag±SD,  microV  RMS| 

2.010.4  | 

1.  710.  8  | 

0.  910.  3  j 

0.  810. 3 

Model-Data  RMSE,  dB  | 

2.  7  | 

3.  1  | 

2.  5  | 

mm, 

Damping  Factor  | 

0.  16  | 

0. 09  | 

0.  19  | 

Max  "Group  Delay",  rasec| 

163  | 

128  1 

40  | 

CEP  Latency±SE,  msec  | 

223138  j 

203147  i 

62112  | 

2413 

R*  of  The  Estimation  | 

0. 64  j 

0.  38  | 

0. 45  | 

0.  72 

CEP  Delay±SE,  msec  | 

60138  | 

75147  | 

22112  1 

1013 

1 

VISUAL  CEP 

MF  REGIONS 

PARAMETERS  | 

THETA  | 

ALPHA  | 

BETA  | 

"40  Hz" 

Frequency  Range,  Hz  | 

5-8  | 

8-12  | 

14-28  | 

Number  of  Cases  (N)  j 

7  | 

7  1 

8  i 

2 

Peak  Frequency±SD,  Hz  | 

5.410.5  | 

10.410.8  | 

20.713.5  | 

Peak  Mag±SD,  microV  RMS| 

2.711.4  | 

1.310.4  | 

1.410.6  j 

Model-Data  RMSE,  dB  | 

2.  3  | 

2.  9  | 

2.  7  | 

Damping  Factor  | 

0.  21  j 

0.  12  | 

0. 21  1 

Max  "Group  Delay",  msec| 

131  | 

106  1 

34  1 

CEP  Latency±SE,  msec  | 

188123  | 

195130  | 

7319  | 

R2  of  The  Estimation  | 

0.  81  | 

0. 56  | 

0.  71  | 

CEP  DelayiSE,  msec  | 

57123  | 

89130  | 

3919  | 
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4.4  Discussion 


4.4.1  CEP  validity 

Before  addressing  the  results,  the  response  (CEP)  must  first  be  proved  to  be  exclu¬ 
sively  and  physiologically  related  to  the  stimulus  (AM  signal).  Based  on  their  equivalent 
filter  models,  the  CCP  TC  ranged  between  40  to  180  msec  (TABLE  II  4.2).  Since  the 
LIA  output  TC  was  much  larger  (10  sec),  the  CCP  vectors  obtained  between  2-3  LIA 
output  TCs  were  assumed  to  be  at  their  "steady-state"  levels.  The  following  supporting 
observations  were  obtained  under  stimulus  on/off  conditions,  while  the  stimulus  MF 
(REF  signal)  and  the  raw  CEP  were  input  to  the  LIA.  Under  stimulus-olf  condition  and 
independently  of  the  ME  value,  only  low  magnitudes  and  random  phases  (status  defined 
as  noise  level)  were  recorded  (clearly  shown  in  the  next  set  of  experiments).  Under 
stimulus-on  condition,  similar  repetitive  stimuli  produced  a  non-random  variable  re¬ 
sponse  vector  (FIGURE  II  4.3).  Also,  since  the  resultant  magnitude  and  phase  MTFs 
were  significantly  different  from  the  recording  system  transfer  functions,  the 
non-ph\Mological-"CCP"  possibility  was  rejected. 

The  most  prominent  peak  was  found  at  the  lowest  MF  scale  (5-8  Hz),  an  active  fre¬ 
quency  region  also  of  the  myogenic  responses  (EMG).  Since  voluntary  muscle  activity 
was  required  only  at  the  end  of  each  attended-to  stimuli  (the  Reaction  Time,  RT.  test), 
and  the  long  I  Sis  (3.2  sec  each)  provided  ample  time  for  the  myogenic  and  related 
neurogenic  transient  potentials  to  die  out,  these  low  MF  potentials  were  concluded  to 
be  genuine  (TPs.  It  was  therefore  concluded  that  the  recorded  CEPs  were  valid,  re¬ 
presenting  neural  activity  exclusively  evoked  by  the  stimuli. 
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4.4.2  Evoked  and  spontaneous  potentials 

The  CEPs  were  extracted  from  the  spontaneous  EEG  and  other  biological  noise 
sources  (EMG,  ECG,  etc.)-  The  spontaneous  EEG,  with  an  equivalent  noise  bandwidth 
similar  to  that  of  the  CEPs  was  considered  to  be  the  primary  noise  source.  SNRs  of 
CEP/spontaneous  EEG  (in  dB  of  mean  square  values)  were  calculated  for  Alpha,  Beta 
and  Theta  frequency  regions.  The  visual  SNRs  (recorded  from  Oz-Al  sites)  were  29,-12 
and  5.7  dB  at  Theta,  Alpha  and  Beta  MF  regions,  respectively  (from  FIGL’REs  II  -4.2, 
4.4).  The  corresponding  auditory  SNRs  (recorded  from  Cz-Al  sites)  were  -2.2,  -14.3  and 
-2.3  dB.  Overall  lower  auditory  CEP  magnitudes  made  their  SNRs  smaller  than  the 
corresponding  visual  SNRs.  The  greatest  inter-sensory  SNR  difference  occurred  at  the 
Theta  frequency  region,  partially  due  to  high  low-frequency  background  activity  picked 
up  at  the  Cz-Al  recording  sites.  The  more  frontal  vertex  electrode  might  have  picked 
up  additional  signals  from  myogenic  sources  (eyeball  and  facial  muscles). 

The  worst  SNRs  were  obtained  in  the  Alpha  frequency  region.  In  contrast  to  the 
dominant  Alpha  EEG  spontaneous  activity,  only  minor  CEPs  were  evoked  at  this  MF 
region.  A  reverse  situation  occurred  at  Beta  and  higher  frequency  regions.  Contrary  to 
distinct  MTF  peaks,  no  visible  EEG  spectrum  peaks  were  detected.  Thus,  as  Spekrcijsc 
et  al.  (1977)  pointed  out  for  the  Beta  MF  region  visual  CEPs,  these  responses  could  not 
have  been  generated  by  spontaneous  EEG-cquivalent  filters. 

Spontaneous  EEG  was  recorded  in  this  current  study  under  opened-eyes,  no-stimulus 
and  no-required  response  conditions.  The  degree  to  which  this  background  "noise '  was 
task-engagement-independent  (an  assumption  made  in  calculating  the  previous  SNRs) 
has  not  been  established  here.  Other  studies  (Kolcs  and  Flor-Hcnry,  1981;  Stcrman, 
1984),  investigating  the  task-engagement  effects  on  the  EEG  Theta  and  Alpha  frequency 
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regions,  provided  mixed  results.  Koles  and  Flor-Henry  (1984)  demonstrated  a  decrease 
of  Alpha  frequency  region  power-density  (the  commonly  accepted  trend)  and  an  increase 
of  Theta/Alpha  frequency  region  power  ratio  (>  1),  in  subjects  engaged  in  verbal  or 
motor  tasks.  Since  motor  responses  were  required  during  the  entire  task-engagement 
sessions,  the  Theta  frequency  region  power-density  was  correctly  attributed  to  myogenic 
responses.  Contrasting  results,  obtained  by  Sterman  (1984),  showed  a  general 
(recording-sitc-dependent  observation)  increase  of  Alpha  frequency  region  power-density 
and  a  decrease  of  Theta/Alpha  frequency  region  power  ratio  (<  1),  in  subjects  engaged 
in  a  flight  simulation  task.  In  summary,  utilizing  the  Lock-In  method,  the  CEP  variance 
(and  not  the  mean  value)  will  reflect  the  recording  SNR,  where  the  noise  itself  (back¬ 
ground  EEC),  like  the  signal  (CEP),  may  be  stimulus-  or  task-dependent. 

4.4.3  CEP  latency  and  delay  estimations 

The  Bode  plots  presented  in  FIGURE  11  4.11  (in  Appendix  A)  should  not  be  per¬ 
ceived  as  traditional  "black-box"  frequency  responses.  The  values  at  each  frequency 
point  represent  a  compound  potential  vector,  averaged  over  multiple,  complexly- 
connected  sources  responding  with  various  latencies.  Therefore,  even  if  linearity  and 
time-invariance  properties  can  be  assumed,  a  singular  neural  delay  and  latency  cannot 
be  calculated  for  the  entire  phase  plot  since  the  phase  and  magnitude  Bode  plots  are  not 
uniquely  related.  However,  if  small  isolated  regions  of  the  Bode  plots  can  justifiably  be 
analyzed  separately,  regional  latency  and  delay  estimations  can  be  obtained. 

This  approach  was  utilized  to  estimate  the  CEP  neural  delays  and  latencies  at  Alpha, 
Beta  and  Theta  MF  regions.  The  following  assumptions  were  made:  CEPs  at  each  MF 
region  were  generated  predominantly  by  a  single,  time-invariant  source;  each  source 
could  be  modeled  by  a  transmission-line  delay  followed  by  a  linear  equivalent  filter, 
partially  disclosed  on  the  Bode  plots  (the  center  frequency  and  the  high  frequency 
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asymptote);  each  equivalent  filter  could  be  modeled  by  a  two  conjugate  pole  LPI  ol 
BPF  with  various  degrees  of  tuning.  These  assumptions  will  be  examined  after  the 
presentation  of  additional  experimental  results.  The  significance  of  the  models'  param¬ 
eters,  summarized  in  TABLE  II  4.2,  is  discussed  in  section  II  4.4.6. 

4.4.4  Modulation  transfer  functions  of  visual  CEPs 

Cross-subject  averages  of  the  fundamental  CEP  component  latencies  at  Theta, 
Alpha,  Beta  and  45-60  Hz  (“40  Hz“)  MF  regions  were  233,  160,  67  and  20  msec,  re¬ 
spectively.  Excluding  the  Theta  MF  region,  these  results  confirm  the  latency  estimations 
of  other  researchers  (Twcel  and  Lunel,  1965;  Regan,  1966,  1972;  Spckreijsc,  1966; 
Spekreijse  et  ah,  1977;  Diamond,  1977;  Junker,  1984;  Junker  and  Pcio,  1984).  When  a 
restricted  (more  reliable)  data  set  was  used  (data  set  used  in  the  neural  delay  estimation, 
TABLE  II  4.2),  the  Theta  MF  region  latency  results  were  also  confirmed.  With  the  ex¬ 
ception  of  the  Beta  MF  region,  the  cross-subject  average  of  magnitude  MTFs  do  not, 
however,  comply  with  the  results  in  the  majority  of  the  above  cited  studies  (reviewed  in 
section  II  4. 1.1. 2-4).  These  differences  will  be  discussed  in  the  following  section. 

The  major  fundamental  magnitude  peak  at  the  Theta  MF  region  was  detected  in  75% 
of  the  visual  MTFs  obtained  in  this  current  study  (FIGURE  II  4.11).  It  was  generated 
for  low  to  medium  average  luminance,  100%  amplitude-modulated,  attended-to,  spot¬ 
light  stimuli.  Magnitude  peaks  at  this  temporal  frequency  range  (Theta)  have  been 
typically  obtained  for  a  high  spatial-frequency,  pattern-reversal  stimulation  method 
(Regan,  1977),  Utilizing  diffused  light  and  unattended  to  stimuli  Spekreijse  (1966), 
Spekreijse  et  al.  (1977)  and  Regan  (1966,  1972)  obtained  predominantly  2nd  harmonic 
component  responses.  The  contrasting  results  of  high  fundamental, '2nd  harmonic  com¬ 
ponent  ratios,  obtained  in  this  current  study  (the  results  of  this  and  the  following  cx- 


peri  merits)  may  be  attributed  to  (1)  a  response-linearizing  phenomenon,  and/or,  (2)  the 
fact  that  the  stimuli  were  attended-to. 

(1)  Utilizing  the  Spckreijsc  "retina-cortex  low-frequency  subsystem"  model  (reviewed 
in  section  II  4. 1.1. 2  ),  the  response  of  the  essentially  non-linear  stage  (a  2nd  harmonic 
generator  modeled  by  an  asymmetric  full-wave  rectifier)  would  have  been  linearized  by 
an  added  high-frequency  auxiliary  signal.  The  auxiliary  signal  could  be  comprised  of 
harmonic  components  generated  in  a  preceding  stage  (non-linear  gain,  magnitude  satu¬ 
ration,  etc.)  or  carried  along  with  the  fundamental  input  signal.  The  large-signal  input 
applied  in  this  current  study  (100%  modulation  depth  stimuli)  might  have  generated 
such  early  higher  harmonic  auxiliary  signals,  linearizing  the  following  rectifier  transfer 
function.  Such  a  linearizing  effect  is  expected  of  fixed  stimulus-rate,  flash  stimuli  input. 
This  input  can  be  regarded  as  a  fundamental  frequency  input  signal  accompanied  by 
auxiliary,  higher  harmonic  components.  And  indeed,  Perry  and  Childers  (1969)  and 
others,  obtained  a  prede  ninant  fundamental  component  TEP  recorded  for  4-6/sec  flash 
stimuli. 

(2)  The  low  MF,  attended-to  stimuli  may  have  been  perceived  as  discrete  flashes 
rather  than  continuous  iignals.  TEPs  in  response  to  flash  stimuli  exhibited  a  major 
magnitude  peak  (peak  VII")  at  a  180-220  msec  latency  (Perry  and  Childers,  1969). 
Therefore,  if  CEP  and  TEP  data  arc  comparable,  the  high  magnitude,  1S8-233  msec 
latency  (TABEEs  II  4  1-’),  Theta  MF  region  CEPs  obtained  in  this  current  study  can 
be  attributed  to  that  P  \  II  TEP  peak.  Atientior.  efTects  are  described  in  the  following 
ATT  experiment. 

The  Alpha  MF  region  peaks,  detected  in  75%  of  the  MTFs  obtained  in  this  current 
study,  were  of  less  significance  when  compared  to  results  of  other  studies  (section  II 
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4.1. 1.2).  These  differences  may  be  attributed  to  (1)  subject's  engagement  effect  and/or 
(2)  the  driven-reference,  CEP  recording  method. 

(1)  Although  not  believed  to  be  totally  unrelated,  the  dependency  between  the  spon¬ 
taneous  EEG  and  the  CEP  magnitudes  at  Alpha  frequency  region  (Alpha  EEG  and 
Alpha  CEP,  respectively),  is  still  controversial  (Childers  and  Perry,  1971;  Spekreijse  et 
al.,  1977  and  others).  If  these  responses  positively  correlate  to  some  degree  (in  peak 
magnitudes  and  frequencies),  task-engagement  that  attenuates  the  Alpha  EEG  magni¬ 
tudes  (see  section  II  4.4.2)  must  decrease  the  Alpha  CEP  response  as  well. 

(2)  A  driven- reference  differential  amplifier  device,  the  recording  system  s  front-end 
amplifier  used  in  this  current  study,  is  characterized  by  a  very  high  CMRR  of  150  dB. 
Therefore  it  is  possible  that  the  widespread  Alpha  EEG  potentials  were  rejected  more 
efficiently  than  in  other  studies  where  conventional  60-80  dB  CMRR  devices  were  used. 

None  of  the  MTFs  obtained  in  this  current  study  showed  the  High  Frequency  MF 
region  peak,  obtained  by  others.  These  low-magnitude  potentials  can  be  generated  only 
for  high  retinal-illuminations  (typically  on  the  order  of  1,000-10,000  troland;  Spekreijse, 
1966;  Regan,  1972),  higher  levels  than  those  utilized  in  this  current  study  (approximately 
1400  troland). 

4.4.5  Modulation  transfer  functions  of  auditory  CEPs 

Cross-subject  averages  of  the  fundamental  CEP  component  latencies  at  Theta, 
Alpha,  Beta  and  "40  Hz"  MF  regions  were  206,  160,  69  and  30  msec,  respectively. 
Magnitude  peaks  at  Theta  (major),  Alpha,  Beta  and  "40  Hz"  frequency  regions  were 
detected  in  approximately  85%,  55%,  85%  and  85%  of  the  individual  MTFs,  respec¬ 
tively  (FIGURE  II  4.11).  In  comparison  to  the  visual  CEPs,  the  auditory  CEPs  were 
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associated  with  lower  signal/noise  magnitude  ratio,  lower  inter-subject  variability  and 
higher  intra-subject  variability. 

CEP  magnitude  and  latency  results  obtained  in  this  study  for  MFs  greater  than  15 
Hz,  roughly  agree  with  those  of  other  studies  (Galambos  et  al.,  1981;  Rees,  1981; 
Rickards  and  Clark,  1984;  Stapells  et  al.,  1984).  (See  review  of  these  studies  in  section 
11  4.1.2).  Since  these  studies  did  not  apply  an  adequate  frequency  resolution  at  lower 
MF  regions,  further  comparison  was  not  meaningful. 

The  degree  to  which  subcortical  neural  structures  (from  the  receptor  level  to  the 
Geniculate  Body  of  the  Thalamus)  contributed  to  the  recorded  compound  CEPs  is  be¬ 
lieved  to  be  negligible.  Each  of  the  "primary'  sensory"  cortical  areas  (Brodmann  17,  42) 
contains  at  least  two-orders  more  excitable  neurons  than  any  subcortical  afferent  nu¬ 
cleus,  with  the  exception  of  the  relatively  remote  retinal  structures. 

4.4.6  CEP  overview 

It  was  lound  that  the  lower  the  MF  of  the  amplitude  modulated  stimuli,  the  larger 
the  CEP  magnitudes  and  the  longer  the  latencies.  This  trend  reflects  the  expected  general 
transfer  function  of  an  afferent  neural  pathway.  Successively  larger  neuron  populations 
generate  larger  compound  potentials  with  lower  frequency  response  and  longer  latencies. 
The  cross-modality  averaged  CEP  latencies  increased  from  35-45  msec  (at  50-60  Hz)  to 
200-240  msec  (at  5-6  Hz). 

Both  auditory  and  visual  magnitude  MTFs  deviated  from  a  monotonic  plot  de¬ 
scription.  showing  distinct  magnitude  peaks  at  non-harmonically-reiated  specific  MF 
regions.  Since  Theta  and  Beta  regions  exhibited  the  most  detectable  and  reliable  re- 
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spon.sc  vectors  in  both  modalities,  they  were  the  center  of  investigation  in  this  and  the 
following  experiments. 

CEPs  at  Beta  MF  region  had  latencies  of  60-70  msec  and  delays  of  30-40  msec. 
Latencies  of  the  initial  compound-potentials  originating  from  the  "primary-sensory' 
auditory  and  visual  cortical  areas  were  found  to  be  10-25  msec  (Regan,  1972;  Celesta. 
1976).  Other  CEP  and  TEP  studies,  analyzing  scalp  potential  distribution,  suggested 
that  responses  with  longer  latencies  (40-70  msec)  were  probably  generated  primarily  at 
the  "secondary-sensory'"  cortical  areas  (Regan,  1972;  Celcsia,  1976;  Spckreipe  ct  al.. 
1977;  Wood  and  Wolpaw,  1982). 

CEPs  at  Theta  MF  region  had  latencies  of  200-240  msec  and  delays  of  50-6<i  msec. 
Other  CEP  and  TEP  studies  suggested  that  such  late-responscs  primarily  originated  from 
multiple  cortical  sources,  overlapping  the  initial  earlier  responses  of  the  primary  and 
secondary  sensory"  cortical  areas  (Spckrcijsc  et  al.,  1977;  Goff  ct  al.,  1 9 7 S ;  Wood  and 
Wolpaw,  1982). 

Auditory  CEPs  at  "40  Hz"  MF  region  had  latencies  of  20-30  msec  and  defies  of  in 
msec.  Latencies  of  the  initial  compound-potentials  originating  from  the  "primary- 
sensory"  auditory  cortical  areas  were  found  to  be  10-25  msec  (Celcsia,  1976).  1  he  decree 
to  which  subcortical  neural  structures  (from  the  receptor  level  to  the  Geniculate  Bodv 
of  the  Thalamus)  contributed  to  the  recorded  compound  CEPs  is  believed  to  be  negligi¬ 
ble.  Each  of  the  "primary  sensory"  cortical  areas  (Brodmann  17,  42)  contains  at  least 
two-orders  more  excitable  neurons  than  any  subcortical  afferent  nucleus,  with  the  ex¬ 
ception  of  the  relatively  remote  retina  structures  (Jung,  1973;  Kay,  1982). 
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The  results  of  the  following  experiments  further  characterize  the  Theta  and  Beta  MF 
region  CUPs  (in  terms  of  response  stability  and  linearity,  intensity  and  attention  effects) 
and  describe  the  auditory-visual  bisensory  CEP  interaction. 
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Chapter  5 

Continuous  Evoked  Potentials  at  Beta  and  Theta 


Modulation  Frequency  Regions 


The  previous  chapter  described  the  general  features  of  the  Continuous  Invoked  Po¬ 
tential  (CEP)  Modulation  Transfer  Functions  (MTFs)  and  characterized  the  most 
responded-to  Modulation  Frequency  (MF)  regions.  The  experiments  described  in  this 
chapter  have  investigated  CEPs  in  response  to  selected  Beta  and  Theta  MF  region 
stimuli,  have  confirmed  previous  results  (MTF  experiment)  and  have  provided  new  re¬ 
sponse  characteristics  and  controls  for  the  following  Auditory- Visual  Interaction  (AxV) 
experiment.  Beta  and  Theta  MF  stimuli  were  selected  for  the  following  reasons:  they 
induce  optimal  CEP  responses  in  both  modalities  (highest  magnitudes,  best  Signal,  Noise 
Ratios,  SNRs,  low  variability);  the  CEPs  were  associated  with  meaningful  delays  and 
latencies  (suggesting  the  cortical  level  as  the  AxV  site). 

In  all  the  following  experiments,  the  raw  magnitude  data  was  expressed  and  manip¬ 
ulated  in  dB  units  instead  of  absolute  RMS  microvolt  units.  This  is  not  an  arbitrary 
choice,  but  rather  an  implementation  of  the  conceptualized  model  of  the  data.  The  in- 
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dependent  variables  tested  in  this  chapter  (stimulus  order,  stimulus  intensity  level,  and 
attention)  and  in  the  following  chapter  (attention,  cross-modality  stimulus  intensity  and 
MF)  were  regarded  as  control  inputs,  modifying  some  parameters  (gain,  filter  sharpness, 
etc.)  of  the  linear  elements  producing  the  CEP  fundamental  responses.  Neither  this 
model  nor  the  alternative  model  (stimulus  and  independent  variables  as  multiple  input) 
could  be  proven  here,  but  the  parametric  control  model  is  more  feasible  physiologically 
and  easier  to  work  with. 
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5.1  DISC  experiment:  CEP  harmonics  and  trends 

5.1.1  Introduction 

5. 1.1.1  Short-term  CEP  trends 

Intra-subject  CEP  magnitude  instability  has  been  attributed  to  many  factors.  Some 
of  them  were  seemingly  controllable  (variations  of  recording  conditions,  myogen.- 
artifacts,  subject  state  and  orientation,  etc.)  and  some  were  not  (noise,  "trends").  Trend 
effect"  implied  a  monotonic  (non-random)  CEP  parameter  variation,  which  was  not  (or 
could  not  be)  related  to  any  of  the  investigated  Independent  Variables  ( I  Vs).  Long-term 
trends  (durations  of  minutes  and  hours),  showed  mostly  a  continuous  decrease  of  CEP 
magnitudes  (Perry  and  Childers,  1966;  Moise,  1980),  referred  to  as  "habituation".  In 
another  study  (Yolton  et  al.,  1983),  mixed  trend  effects  (increasing  or  decreasing  mag¬ 
nitudes)  and  noise  effect  accounted  for  25%  and  50%  of  the  total  intra-subject  CEP 
magnitude  variability,  respectively. 

Since  long  stimulus  durations  were  used  in  the  AxV  experiment  (90-1U0  sec),  controls 
of  short-term  CEP  trends  had  to  be  established  first.  The  DISC  ("DISCRETE")  exper¬ 
iment  provided  parameter  trends  of  CEPs  recorded  under  maintained  attention  level 
during  the  first  90-100  sec  past  stimulus  initiation. 

5. 1.1. 2  1st  and  2nd  harmonics  of  CEPs 

Latencies  and  delays  were  estimated  in  the  previous  chapter  from  the  phase  MTFs. 
assuming  a  linear  minimum-phase  system  plus  a  fixed  delay.  The  validity  of  the  linearity 
assumption  is  tested  in  this  experiment  where  the  CEPs  1st  and  2nd  harmonic  compo¬ 
nents  are  evaluated. 
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5.1.2  Methodology 


The  four  consecutive,  most  responded-to  frequency  values  in  each  of  the  Beta  and 
Theta  Ml-  regions  (magnitude  peak  frequencies)  were  tested  for  every  subject,  a  total 
of  16  audio  and  visual  discrete  MFs  (4  "mini"  MTFs).  Within  the  audio  or  visual  series, 
MFs  were  alternatively  sequenced  between  Beta  and  Theta  MF  regions.  Each  run  con¬ 
sisted  of  a  twice-repeated  pattern  of  three  repetitive,  single  MF  stimuli.  Average  dura¬ 
tions  of  stimulus-on,  Inter  Stimulus  Interval  ( I S I)  and  Inter  Pattern  Interval  ( I  PI )  were 
32.5,  1.4  and  32.5  sec,  respectively.  CEPs  were  recorded  under  the  same  attentiveness 
(Reaction  Time  test,  RT)  and  stimulus  intensity  ("0"  dB)  conditions  utilized  in  the  pre¬ 
vious  MTF  experiment.  The  DISC  experiment  recording  session,  audio  first  and  visual 
second,  lasted  three  hours,  including  rest  breaks.  For  more  details,  refer  to  TABLE  I 
3.3. 

The  DISC  experiment  data  set  was  processed  with  a  Lock-In  Amplifier  ( L I A )  output 
Time-Constant  (TC)  of  10  sec.  CEP  magnitudes  and  phases  of  sequentially  corre¬ 
sponding  stimuli  were  cross-pattern  averaged  for  each  run.  Since  trend  effects  within  a 
time-frame  of  90-100  sec  were  the  focal  point  of  this  investigation,  the  inter-pattern 
variability  was  regarded  as  noise.  Relative  magnitude  (dB)  and  phase  (degrees)  values 
(relative  to  the  the  run's  averaged  values)  w'erc  used.  The  resultant  1st.  2nd  and  3rd  CLP 
vectcs  (ORDER  classes)  w'ere  tested  for  response  stability. 

In  order  to  extract  the  CEP  2nd  harmonic  vectors,  the  DISC  data  set  was  off-line 
re-demodulated  with  a  reference  signal  having  twice  the  stimulus  MF.  For  this  analysis, 
the  fundamental  and  2nd  harmonic  component  vectors  were  obtained  bv  cross-stimulus 
averaging  of  magnitude  and  phase  values  in  each  run  (run  average  values).  For  each 
group  of  Ml  (Beta,  Theta)  x  signal  (audio,  visual)  x  subject,  the  most  responded-to  MF 


was  identified  (peak  magnitude  and  frequency).  The  latency  and  its  reliability  level  ol 
the  CEPs  recorded  for  that  MF,  were  estimated  from  regression  and  correlation  coeffi¬ 
cients,  respectively  (computed  on  a  3  or  4  phase-point-wide  window). 

5.1.3  Results 

5. 1.3.1  Raw  data  example 

An  example  of  LI  A  output  of  a  complete  run  is  presented  in  FIGURE  11  5.1.  Hie 
1st  harmonic  vector  was  obtained  with  LI  A  output  TCs  of  1  and  10  sec,  and  the  2nd 
harmonic  vector  with  an  LIA  output  TC  of  10  sec.  Note  the  different  magnitude  scales 
and  the  relatively  "noisier"  2nd  harmonic  response.  It  is  also  evident  that  the  TC  of  the 
neural  response  was  much  smaller  than  the  10  sec  LIA  output  TC. 

5. 1. 3. 2  CEP  stability 

CEP  variations  over  time  are  presented  in  FIGURE  II  5.2  and  tested  for  significance 
in  I  ABLE  II  5.1.  In  light  of  the  large  inter-subject  variability,  the  average  trends  in  each 
plot  arc  supplemented  with  the  individual  data  for  comparison.  In  general,  a  significant 
trend  was  detected  only  for  the  magnitudes  (P  >  F  of  0.006).  CEP  magnitudes  tended 
to  increase  with  time  (an  average  increment  of  15%  over  a  time  period  of  90-100  see), 
significantly  at  audio  Beta  and  visual  Theta  MF  regions  (P  >  F  of  0.03).  CEP  phases 
showed  only  non-sigruficant  mixed  trends  at  audio  Theta  ( increase)  and  visual  Beta  (de¬ 
crease!  Mb'  regions. 


-74- 


FIGURE  II  5.  1  :  VISUAL  CEP,  SUBJECT  HR,  35  SECONDS  AVERAGED 
STIMULUS-ON  DURATION,  5.5  Hz  MODULATION 
FREQUENCY.  M:  CEP  MAGNITUDE,  P:  CEP  PHASE, 
TCs:  LOCK-IN  AMPLIFIER  PRE-  AND  OUTPUT 
AMPLIFIER  TIME -CONSTANTS,  STIMULUS-OFF 
TRANSITION:  REACTION-TIME  CLUE. 


AUDITORY  AND  VISUAL  CEP  TRENDS 


TABLE  II  5.1  :  Trend  significance  between  ORDER  groups.  MEAN  :  Average 
across  N  observations,  ORDER  :  1st,  2nd  and  3rd  sequential  CEPs  reduced  by  the 
run's  mean  value,  NS  :  Not  Significant,  P  >  F  of  >0.2,  MF  :  Modulation  I  re- 
quency. 


5. 1.3. 3  CEP  harmonics 


1st  and  2nd  harmonic  comparison  of  CEP  peak  absolute  magnitudes  and  latencies  is 
presented  in  TABLE  II  5.2.  The  2nd  harmonic  magnitudes  were  significantlv  smaller 
than  the  fundamental  magnitudes  (approximately  43%  of  the  fundamental  magnitudes). 
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STABILITY  OF  CEP  MAGNITUDES  AND  PHASES 

SIG=AUDI0  FREQ=BETA 


FIRST  SECOND  THIRD 

SEQUENCE  OF  PATTERN  STIMULI 


FIGURE  H  5.2  :  MAGNITUDE  GAIN  (DB)  AND  PHASE  SHIFT  (DEGREES)  RELATIVE  TO  THE 
VALUES  OF  THE  AVERAGED  1ST,  2ND  AND  3RD  SEQUENTIAL  STIMULI. 
NUMBERS:  INDIVIDUAL  SUBJECTS,  THICK  LINE:  MEAN  OF  9  SUBJECTS. 


FIRST 


SECOND 

SEQUENCE  OF  PATTERN  STIMULI 


THIRD 


FIGURE  II  6.Z  : 


MAGNITUDE  GAIN  (DB)  AND  PHASE  SHUT  (DEGREES)  RELATIVE  TO  THE 
VALUES  Or  THE  AVERAGED  1ST,  2ND  AND  3RD  SEQUENTIAL  STIMULI. 
NUMBERS:  INDIVIDUAL  SUBJECTS,  THICK  LINE:  MEAN  OF  9  SUBJECTS. 
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STABILITY  OF  CEP  MAGNITUDES  AND  PHASES 

S I G=V I SUAL  FREQ=BETA 


SEQUENCE  OF  PATTERN  STIMULI 


FIGURE  II  5.2  :  MAGNITUDE  GAIN  (DB)  AND  PHASE  SHIFT  (DEGREES)  RELATIVE  TO  THE 
VALUES  OF  THE  AVERAGED  1ST,  2ND  AND  3RD  SEQUENTIAL  STIMULI. 
NUMBERS:  INDIVIDUAL  SUBJECTS,  THICK  LINE:  MEAN  OF  »  SUBJECTS. 


-79- 


wmmsomo  z—  ni/i>nj  coo  z—  roch-zc»5: 


STABILITY  OF  CEP  MAGNITUDES  AND  PHASES 

SIG-VISUAL  FREQ*THETA 


FIRST  SECOND  THIRD 


SEQUENCE  OF  PATTERN  STIMULI 


FIGURE  n  5.2  :  MAGNITUDE  GADI  (OB)  AND  PHASE  SHUT  (DEGREES)  RELATIVE  TO  THE 
VALUES  OF  THE  AVERAGED  1ST,  2ND  AND  3RD  SEQUENTIAL  STIMULI. 
NUMBERS:  INDIVIDUAL  SUBJECTS,  THICK  LINE:  MEAN  OF  9  SUBJECTS. 
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The  latencies  of  the  CEP  harmonics  did  not  significantly  differ,  although  a  trend  of 
shorter  auditory  CEP  2nd  harmonic  latencies  was  observed  (predominantly  at  Beta  ME 
region). 


CEP  FUNDAMENTAL  AND  2ND  HARMONIC  COMPARISON 


TABLE  II  5.2  :  t-test  of  CEP  harmonic  components.  MEAN  :  averaged  value  over 
N  observations,  SD  :  Standard  Deviation,  R  :  correlation  coefficient  of  the  estimated 
latencv,  BETA  and  THETA  :  modulation  frequency  regions,  NS  :  Not  Significant, 
P  >  |t|  of  >  0.2. 


PARAMETER 

j  FUNDAMENTAL 

2nd  HARMONIC 

| t-TEST| 

1  1 

Magnitudes  in 

|  N |  MEANISD 

IN 

|  MEANiSD 

ip  >itn 

microvolt  RMS 

1  1 

i  i 

i  i 

Auditory  BETA 

|  9  |  0.62+0.35 

|  9 

|  0.2610.13 

1  "1 
|0.015  | 

Auditory  THETA 

1 9 }  1.6910.49 

|9 

|  0.  6810. 37 

|0.  0002| 

1 _ l 

Visual  BETA 

|  8  |  0.8410.5  7 

|  8 

|  0.3410.14 

i  —  i 

|0.04  | 

Visual  THETA 

|  8  j  1.4911.28 

|  8 

|  0.7210.48 

1  0. 14| 

|  1 

Latencies 

|  N | MEAN1SD  |  R 

|  N 

|  MEANISD  |  R 

IP  > 1 t | | 

in  msec 

1  1  1 

1  1  1 

1  1 

1  1 

Auditory  BETA 

II  1  " 

|  9 |  931102  |0.94 

|  6 

1  1 
|  361  16  |0. 90 

1  1 

1  0.14| 

Auditory  THETA 

| 9 | 18 1±  73  |0. 95 

i. 1 _ 1 _ 

|6 

j 166±108  |0.95 

i _  _ i 

1  NS  | 

Visual  BETA 

ii  i 

|  9 |  881  20  jO. 97 

|  8 

|  831  16  jO. 98 

1 - 1 

1  NS  j 

Visual  THETA 

| 9 | 19 11  57  |0. 98 

j  8 

| 1921  53  |0. 97 

|  NS  j 

5.1.4  Discussion 

5. 1. 4. 1  CEP  trends 


The  general  trend  of  a  15%  average  increase  in  magnitude  was  evenly  distributed  over 
the  90-100  sec  trial  duration.  It  was  not  accompanied  by  any  equivalent  phase  trend  nor 
was  it  significantly  associated  with  a  particular  ME  region  or  modalitv.  With  an  output 
1C  o!  10  sec.  t  single  pole  I  PE),  the  LI  A  magnitude  response  to  a  step-function  input 
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was  higher  than  90%,  99%  and  99.9%  of  the  full-scale  magnitude  after  30.  and  9o 
sec,  respectively,  a  total  of  1 1%  increase,  most  of  it  obtained  after  only  6<i  sec.  Since  the 
observed  experimental  magnitude  trend  was  variable  and  different  from  the  expected 
LIA  response  (to  a  step-function  input),  it  is  believed  to  be  a  predominantly  physiolog¬ 
ical  phenomenon. 

Short-term  magnitude  trends  (within  the  first  90  sec  of  the  recorded  data)  were  in¬ 
vestigated  in  only  a  few  studies  (Perry  and  Childers,  1966;  Regan,  1966;  Yolton  et  a!.. 
1983).  Visual  CEP  magnitude  trends  were  obtained  from  time  periods  of  30-240  sec 
(Perry  and  Childers*  1966)  and  30-70  sec  (Yolton  et  al.,  1983)  after  stimulus  initiation, 
avoiding  initial  high  magnitude  transient  responses.  Perry  and  Childers  used  1500  lux. 
2.2  degree  visual  angle,  4  Hz  flash  stimuli.  Regan  used  9000  troland,  14  degree  stimulus 
field,  spotlight  stimuli  modulated  at  15.5  Hz.  Yolton  et  al.  used  102.8/15.4  cd  m2 
(bright/dark  checks),  checkerboard  pattern  stimuli  with  an  alternation  reversal  rate  of 
15  Hz.  All  these  studies  found  that  the  CEP  magnitudes  recorded  approximately  30  sec 
after  stimulus  initiation  were  constant  and  stable  over  the  trial  duration.  Subject 
attentiveness,  uncontrolled  in  any  of  the  above  cited  studies,  might  explain  the  apparent 
contradiction  regarding  the  significant  trend  observed  in  this  current  study.  While  per¬ 
forming  the  RT  test,  the  subjects  could  gradually  "tune  in"  to  the  stimulus  duration  (al¬ 
though  it  was  partially  randomized),  improving  their  scores  and  possibly  increasing  their 
CEP  magnitudes  in  the  process. 

Regan  (1966)  described  an  initial  transient  CEP  magnitude  increase  that  occurred  in 
some  subjects  during  the  first  20  sec  of  the  response.  Although  it  may  have  occurred  in 
some  of  the  runs,  the  general  CEP  magnitude  trend  in  this  present  study  did  not  reveal 
such  an  initial  transient  response.  The  first  30  sec  of  the  CEPs  were  characterized  bv  the 
smallest  magnitudes  (LIA  TC-indepcndent). 


-82- 


The  CUP  temporal  trends  observed  in  this  experiment  could  not  be  predicted  from  the 
simple  model  of  delay-line  followed  by  a  linear  tuned  filter  (assumed  m  this,  and  many 
other  studies).  Practically,  it  implies  that  the  so-called  "steady-state"  response  could  be 
assumed  onlv  if  it  is  referred-to  within  a  defined  time-frame  past  the  stimulus  onset. 

5. 1.4. 2  CEP  harmonics 

Auditory  and  visual  CEP  magnitudes  obtained  at  Beta  and  Theta  MF  regions  were 
predominantly  composed  of  fundamental  components.  Rodenburg  ct  al.  (1972)  and 
Rees  (1981)  demonstrated  similar  auditory  CEP  results  at  Theta  M  F  region  (4  Hz),  using 
stimuli  of  Amplitude  Modulated  (AM  )  noise  and  AM  1  KHz  tone  stimuli,  respectively. 
Comparable  analysis  of  the  auditory  Beta  MF  region  was  not  available. 

In  the  visual  system,  Spckreijsc  ct  al.  (1977)  characterized  the  Beta  MF  region  CEPs 
as  highly  linear  and  relatively  distortionless  responses.  On  the  other  hand  (and  contrary 
to  the  findings  of  this  current  study),  their  Theta  MF  region  CEPs  were  highly  non¬ 
linear,  comprised  predominantly  of  the  2nd  harmonic  component  (sec  section  II  4. 1.1. 2). 
These  differences  were  addressed  in  section  II  4.4.4. 

In  conclusion,  the  CEPs  recorded  in  this  study,  in  response  to  sinusoidally 
amplitude-modulated  signals,  were  predominantly  composed  of  fundamental  compo¬ 
nents  (averaged  fundamental '2nd  harmonic  magnitude  ratio  of  0.43).  However  inaccu¬ 
rate  (due  to  temporal  trends,  and  fundamental  magnitude  saturation  discussed  in  the 
next  experiment),  the  CEP  was  predominantly  linear.  More  research  is  needed  to  de¬ 
termine  the  type  and  location  of  the  non-linear  elements. 
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5.2  INT  experiment:  stimulus  intensity  level  effect 

5.2.1  Introduction 

The  gain  of  a  linear  system  is  independent  of  the  input  signal  magnitude.  In  general, 
this  is  not  the  case  in  the  auditory  and  visual  systems  where  Evoked  Potential  (EP) 
magnitude-gains  and  latencies  were  found  to  be  stimulus  intensity-level  dependent. 
Since  the  intensity  variable  was  regarded  in  this  study  as  a  control  input  (i.e.  the  AxV 
experiment),  its  effect  had  to  be  determined. 

In  a  pilot  study,  auditorv  and  visual  CEPs  were  obtained  for  the  highest  stimulus  in¬ 
tensity,  tolerated  (audio)  or  producible  (visual,  with  the  available  instrumentation),  in 
order  to  maximize  their  magnitudes  and  SNRs.  These  "0"  dB  levels  were  108  dB  SP1. 
and  equivalent  '•ource  illumination  of  44  lux  for  the  audio  and  visual  stimuli,  respec¬ 
tively.  The  [NT  experiment  characterized  the  response  vector  in  thc-"0"  to  -24  dB 
stimulus  intensity  range. 

5.2.2  Methodology 

The  most  responded-to  MFs,  a  total  of  4  audio  and  visual  discrete  Beta  and  Theta 
MFs  were  tested  for  each  subject.  Intensity  level  was  increased  from  -24  dB  to  ”<f  dB 
in  6  dB  increments,  maintaining  a  100%  modulation  depth  and  constant  background 
level.  Each  run  was  comprised  of  a  series  of  15  stimuli,  with  a  repetition  of  3  stimuli  per 
intensity  level.  Average  Stimulus  duration  (S)  and  ISI  were  32.5  and  1.4  sec,  respec¬ 
tively.  CEPs  were  recorded  under  attentiveness  conditions  (RT  test,  similar  to  the  MTF 
experiment),  and  the  entire  INT  recording  session,  audio  first  and  visual  second,  lasted 
one  hour,  including  rest  breaks.  For  more  details,  refer  to  TABEE  1  3.3. 
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5.2.3  Results 


Compensated  CEP  magnitude  (dB)  and  phase  values  (relative  to  the  run's  mean 
value)  were  plotted  against  stimulus  intensity  levels  in  FIGURE  II  5.3.  In  light  of  the 
large  inter-subject  variability,  each  trend  (average  values'  regression-line)  was  supple¬ 
mented  by  the  individual  data  for  comparison.  Statistical  significance  of  the  trends  is 
presented  in  TABLE  II  5.3,  where  CEP  vectors  obtained  at  Low  (-24,  -18  dB)  and  High 
(-6,  "0"  dB)  stimulus  intensity  levels  were  compared. 

In  general,  an  increase  in  stimulus  intensity  was  significantly  associated  with  visual 
CEP  magnitude  increase  and  visual  CEP  Beta  MF  region  phase  lead.  The  dynamic 
range  of  the  visual  Beta  MF  region  CEP  magnitudes  (from  noise  level  to  maximal  mag¬ 
nitude  level)  was  fully  exposed  by  the  stimulus  intensity  range,  while  the  Theta  MF  re¬ 
gion  dynamic  range  was  not.  In  both  MF  regions  the  "0"  and  -12  dB  intensity  levels 
evoked  significantly  different  magnitudes. 

Common  significant  trends  were  not  observed  in  the  auditory  CEPs.  The  cross- 
subject  average  magnitude  and  phase  were  generally  unaffected  by  the  stimulus  intensity 
level,  although  within  subject,  mixed  trends  were  observed. 
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CEP  MAGNITUDE  AND  PHASE  VERSUS  STIMULUS  INTENSITY 

S I G=AUD 1 0  FREQ=BETA 


|  i  i  i  i  i  |  i  l  i'I  I  |  I  I  I  I  I  [  I  |  I  I  I  | 

-24  -IB  -12  -6  "0" 

STIMULUS  INTENSITY  IN  OB 


FIGURE  0  5.3  :  MAGNITUDE  GAIN  (DB)  AND  PHASE  SHIFT  (DEGREES)  RELATIVE 
TO  THE  CROSS-INTENSITY  AVERAGED  VALUES.  NUMBERS: 
INDIVIDUAL  SUBJECTS,  DOTS:  AVERAGES  OF  8  SUBJECTS, 

SOLID  LINE:  LINEAR  REGRESSION  LINE  FITTED  TO  THE  AVERAGES. 
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CEP  MAGNITUDE  AND  PHASE  VERSUS  STIMULUS  INTENSITY 

SIG-AUDIO  FREQ=THETA 


STIMULUS  INTENSITY  IN  OB 


FIGURED  5.3  :  MAGNITUDE  GAIN  (DB)  AND  PHASE  SHUT  (DECREES)  RELATIVE 
TO  THE  CROSS-INTENSITY  AVERAGED  VALUES.  NUMBERS: 
INDIVIDUAL  SUBJECTS,  DOTS:  AVERAGES  OF  S  SUBJECTS, 

SOLID  LINE:  LINEAR  REGRESSION  LINE  nTTED  TO  THE  AVERAGES. 
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CEP  MAGNITUDE  AND  PHASE  VERSUS  STIMULUS  INTENSITY 

S1G-VISUAL  FREQ=BETA 
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CEP  MAGNITUDE  AND  PHASE  VERSUS  STIMULUS  INTENSITY 

SIG=V I SUAL  FREQ=THETA 


-24  -18  -12  -6  "0" 

STIMULUS  INTENSITY  IN  DS 


FIGURE  D  5.3  :  MAGNITUDE  GAIN  (DB)  AND  PHASE  SHUT  (DEGREES)  RELATIVE 
TO  THE  CROSS— INTENSITY  AVERAGED  VALUES.  NUMBERS: 
INDIVIDUAL  SUBJECTS,  DOTS:  AVERAGES  OF  8  SUBJECTS, 

SOLID  LINE:  LINEAR  REGRESSION  LINE  FITTED  TO  THE  AVERAGES. 
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INTENSITY  EFFECT  ON  AUDITORY  AND  VISUAL  C  EPs 


TABLE  II  5.3  :  t-test  between  "High"  and  "Low"  CEP  groups.  "High"  :  (  0",  -6  d B ) 
INTcnsity  levels,  "Low"  :  (-18,  -24  dB)  INTcnsity  levels,  MEAN  :  average  across  N 
observations  reduced  by  the  run's  mean  value,  SD  :  Standard  Deviation.  MI  : 
Modulation  Frequency,  NS  :  Not  Significant,  P  >  F  of  >  0.2. 


PARAMETERS  | 

_  i 

AUDITORY  CEP 

VISUAL  CEP 

| 

BETA  MF  | 

— 

THETA  MF 

BETA  MF 

THETA  MF 

MAGNITUDES  | 

1 

(in  dB)  1 

- 1 

1 

-  ...  | 

INT  groups | 

- 1 

1 

MEAN±SD  :  | 

1 

"High"  | 

0.2112.61  | 

-0. 4511. 29 

3.  6211.  55 

2.  6312.  50 

"Low”  j 

_ _ i 

-0.  37±2.40  1 

0.  3112.  00 

-3. 6112. 20 

-2. 5212. 33 

INT  groups | 

- 1 

i 

t-test:  1 

i 

N  (group)  j 

16  | 

16 

16 

16 

P  >|t|  | 

NS  1 

NS 

0. 0001 

— 

0.  0001 

PHASES  (in  | 

1 

degrees)  | 

i 

1 

■■■ 

i 

INT  groups | 

- 1 

1 

— 

MEAN+SD  :  | 

1 

"High”  | 

-5.  7123. 6  | 

1.  11  8.  7 

18. 7123. 0 

0.  1132. 4 

"Low"  | 

_ 1 

2.  4±25. 7  | 

-2.  0+  9.  5 

-17.4133.4 

9.  1136. 0 

1 

INT  groups | 

i 

t-test:  j 

i 

N  (group)  | 

16  | 

16 

16 

16 

P  >|t|  | 

NS  | 

NS 

0.  005 

NS 

5.2.4  Discussion 


With  the  current  hardware  setup,  only  the  fundamental  and  the  2nd  harmonic  CEP 
components  could  be  investigated.  In  the  DISC  experiment,  high  average 
lundamcntal/2nd  harmonic  magnitude  ratios  (of  2.4,  TABLE  II  5.2)  were  obtained  even 
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at  the  highest  stimulus  intensity  level  ("0"  dB).  The  1NT  experiment  was  designed  to 
explore  only  the  fundamental  CEP  component. 

Due  to  the  small  CEP  magnitudes  and  poor  SNRs  obtained  in  the  pilot  study  (under 
"0"  dB  stimulus  level),  a  stimulus  intensity  range  of  "0“  to  -24  dB  was  thought  to  be  ad¬ 
equate  to  fully  expose  the  CEP  magnitude  dynamic  range.  This  intensity  range  was  ad¬ 
equate  for  the  visual  CEPs,  but  insufficient  for  the  auditory  CEPs.  The 
cross-subjcct-averaged  CEP  magnitude  seemed  to  be  stimulus  intensity-independent. 
This  phenomenon,  the  Battening  of  the  fundamental  CEP  magnitude  component  (plot¬ 
ted  versus  intensity),  is  referred  to  here,  and  in  the  following  cited  studies,  as  "satu¬ 
ration  .  1  he  nature  of  this  saturation  ("hard"  or  "soft")  could  not  be  determined  here 
due  to  the  insufficient  intensity  scale  resolution  (6  dB  decrements). 

When  the  audio  stimulus  intensity  range  of  84-108  dB  SPL  was  tested,  the  auditory 
CLP  magnitude  averages  of  both  ML  regions  were  flat,  probably  completely  saturated 
(FIGURE  II  5.3).  Rees  (1981)  recorded  CEPs  in  response  to  1  KHz  tone,  100%  AM 
by  4  II/  ML  stimuli  (Theta  ME  region),  administered  binaurally  at  0-80  dB  SL.  He 
demonstrated  magnitude  saturation  (in  two  subjects)  at  stimulus  intensity  levels  exceed¬ 
ing  50  dB  SL.  Rickards  and  Clark  (1984)  recorded  CEPs  in  response  to  500  Hz  tone, 
94° o  AM  by  24  Hz  ME  stimuli  (Beta  MF  region),  ipsilatcraUv  administered  ar  0-100  dB 
SPL.  They  obtained  a  magnitude  increase  of  20  dB  and  a  phase-lead  of  60  aegrees  over 
the  50-100  dB  SPL  stimulus  intensity  range. 

Results  difference  between  Rickards  and  Clark  (1984)  study  and  this  study  might  be 
attributed  to  the  different  carrier  frequencies  utilized.  The  saturation  phenomenon  ob¬ 
served  in  this  study  might  be  attributed  to  the  high  modulation  depth  utilized  (100%), 
instead  of  to  the  stimulus  averaged  intensity.  Other  studies  showed  that  CEP  magnitude 


saturation  occurred  at  modulation  depths  exceeding  10-50%  (Spekreijse,  1066:  Regan, 
1972;  Rodenburg  et  al.,  19721. 

To  summarize,  at  "0"  dB,  both  auditory  and  visual  CEP  magnitudes  were  saturated, 
the  auditory  more  profoundly  Apparently  related  to  this  fact,  only  the  visual  CEPs 
showed  significant  differences  between  "0”  and  -12  dB  stimulus  intensity  levels.  Since 
these  two  intcncity  levels  were  utilized  in  the  AxV  experiment,  no  inter-modality  inten¬ 
sity  effect  should  be  detected  there  for  the  auditory  CEPs. 
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5.3  A  TT  experiment:  subject  attentiveness  effect 

5.3.1  Introduction 

The  elfcct  of  subject  attentiveness  on  Transient  Evoked  Potentials  (TEPs)  has  been 
extensively  investigated.  In  general,  attcndcd-to  stimuli  evoked  higher  EP  magnitudes 
(Squires  ct  al.,  1977;  Hillyard  et  al.,  1978).  Many  models  were  constructed  to  define  at¬ 
tentive  behavior  and  suggest  the  underlying  mental  processes.  An  early  stage  of  "signal 
recognition"  was  related  to  variations  of  50-75  msec  latency,  TEP  magnitude  compo¬ 
nents.  This  process  required  a  simple  physical  cue  and  high  stimulus  rate,  and  was  en¬ 
hanced  by  low  intensity  level  stimulation  (Hillyard  et  al.,  1978).  Attention  allocation 
could  be  controlled  by  more  discriminablc,  inter-modality  cues  (Donchin  and  Cohen, 
1967;  Hartley,  1970;  Tord  et  al..  1973;  Schechter  and  Buchsbaum,  1973).  The  behavioral 
parameters  actually  being  measured  in  these  cited  studies  were  debatable.  It  can  be  ar¬ 
gued  that  the  TEP  stimulation  procedure  (requiring  low  frequency  clicks  or  (lashes)  and 
the  necessity  to  "mix"  the  relevant  stimuli  (rcacted-upon  cues)  with  other  "inert"  stimuli, 
the  so-called  "attended-to"  TEPs  mostly  reflected  the  recognition  process  of  the  transient 
target  clue. 

The  purpose  of  the  ATT  experiment  in  this  current  study  was  to  investigate  the  CEP 
vector  recorded  in  response  to  attended-to  versus  non-attended-to  continuous  stimuli. 
Attention  allocation  was  controlled  by  inter-modality  simple  cues  with  an  a-priori 
known  sequence.  In  this  experiment,  the  recognition  process  of  the  transient  cues  could 
not  interfere  with  the  attention  allocation  effect. 

5.3.2  Methodology 


In  this  experiment,  each  run  consisted  of  simultaneously  presented  and  only  mar- 


ginally  overlapping,  audio  and  visual  series,  each  consisting  of  4  repetitive  stimuli.  Av¬ 
erage  Stimulus  duration  (S),  1SI  and  inter-modalitv  Audio-Visual  Delay  <AVD\  all  were 
32.5  sec.  Since  the  stimulus  durations  were  randomized  (10%  and  25°  o  of  Beta  and 
Theta  MF  region  stimulus  duration,  respectively),  the  inter-modality  sequence  of  stim¬ 
ulus  gate  transitions  (during  the  short  time  when  both  audio  and  visual  stimuli  were 
overlapping)  appeared  to  be  random  too.  Each  run  utilized  a  single  MF  region  and  Low 
(-12  dB)  or  High  ("O'  dB)  stimulus  intensity  levels.  Beta  and  Theta  MFs  were  similar  to 
the  optimal  frequencies  previously  determined  in  the  DISC  experiment;  only  one  fre¬ 
quency  bin  differentiated  between  the  audio  and  visual  MFs  within  the  same  MF  region 
(average  frequency  difference  of  9%).  Therefore  both  sensory  channels  were  stimulated 
by  similar  and  nearly  optimal  MFs.  Attention  was  controlled  by  instructing  and  training 
the  subjects  to  respond  to  the  following  clues  (RT  test):  stimulus  'on"  and  "off'  transi¬ 
tions  were  reacted  upon  in  the  first  and  last  two  stimuli  in  each  series,  respectively.  The 
auditory  series  always  led  and  its  third  stimulus  required  both  "on"  and  "ofT  responses. 
Consequently,  CEPs  were  recorded  in  response  to  unattended-to  and  attended-to  stimuli 
under  cross-modality  and  intra-modality  attention  allocation  conditions,  respectively. 
The  entire  ATT  recording  session  lasted  one  hour,  including  training  and  rest  breaks. 
For  more  details,  refer  to  TABLE  I  3.3. 

5J.3  Results 

The  results  of  magnitude  ratios  (in  dB)  and  phase  differences  (of  CEPs  in  response 
to  attended-to  versus  unattended-to  stimuli)  were  plotted  against  stimulus  categories 
(signal-MF  region-intensity)  in  FIGURE  II  5.4.  In  light  of  the  large  inter-subject  vari¬ 
ability,  the  trends  (average  values)  were  supplemented  with  the  individual  data  for  com¬ 
parison.  Statistical  significance  of  the  attention  effect  was  tested  on  the  visual  CEP 
magnitude  (in  dB)  and  phase  (in  degrees)  compensated  data  (relative  to  the  run  s  mean 
values,  TABLE  II  5.4). 
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Significant  trends  of  higher  magnitudes  recorded  in  response  to  attended-to  stimuli 
were  observed  only  for  the  visual  CEPs.  The  attention  effect  on  visual  CEPs  was  asso¬ 
ciated  with  unequal  average  magnitude-gains  of  4.70  and  2.76  dB  for  the  High  and  Low 
stimulus  intensity  levels,  respectively.  A  significant  visual  CEP  phase-lead  was  recorded 
in  response  to  Beta  MF  region,  low  intensity  level  (-12  dB),  attended-to  stimuli.  The 
auditory  CEPs  did  not  reveal  any  significant  trends. 

5.3.4  Discussion 

Attention  effect  was  found  only  in  visual  CEPs  where  attended-to  stimuli  elicited 
higher  magnitudes.  -Fagan  et  al.  (1984),  recording  CEPs  in  response  to  a  15.6  Hz  phase 
reversal  rate  checkerboard  stimulus,  found  a  similar,  statistically  non-significant  trend. 
They  obtained  higher  average  visual  CEP  magnitudes  under  hypnotically-induced  at¬ 
tention  condition.  Their  ill-defined  control  session,  during  which  subjects'  attention  was 
not  directed  or  verified,  could  have  prevented  them  from  obtaining  more  conclusive  re¬ 
sults. 

In  this  study,  similar  attention-related  magnitude  gains  were  detected  independently 
in  early  (Beta,  75-85  msec  latency)  and  late  (Theta,  165-210  msec)  visual  CEP  responses 
(TABLE  II  4.2).  Inter-MF  region  correlation  of  the  attention-related  magnitude  gains 
did  not  reveal  any  dependency  (non-significant  correlation  coefficients  of  R  <  0. 1).  This 
observation  may  suggest  that  attention  allocation  effects  are  independently  induced  at 
various  levels  of  the  visual  system. 
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ATTENTION  EFFECTS  ON  CEP  MAGNITUDES 


ABH  ABl  ATH  AIL  VBH  VBL  VTH  VTL 


CATEGORIES  (SIG  FREQ  I  NT) 

FIOURS  □  B  AA  :  IIAONTTUDI  RATIOS  Hi  DB  (ATTKHCSD  /  NOT  ATTENDED), 

NUMBERS:  INDIVIDUAL  SUBJECTS,  BARS:  AVERAGES  OF  8  SUBIKCTS. 
COOKS:  AUDITORY  (A)  AND  VISUAL  (?)  C VT,  BETA  (B)  AND  THBTA  (T) 
MODULATION  FREQUENCIES.  HIGH  (H)  AND  LOW  (L)  INTENSITY  LEVELS. 
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ATTENTION  EFFECTS  ON  CEP  PHASES 
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-90 

-135 

ABH  ABl  ATH  ATI  VBH  VBL  VTH  VTL 
CATEGORIES  (SIC  FREQ  I  NT) 

FIGURE  n  S.4B  :  PHASE  DIFFERENCES  IN  DEGREES  (ATTENDED  -  NOT  ATTENDED). 

NUMBERS:  INDIVIDUAL  SUBJECTS,  BARS:  AVERAGES  OF  fl  SUBJECTS. 
CODES:  AUDITORY  (A)  AND  VISUAL  (V)  CEP'S,  BETA  (B)  AND  THETA  (T) 
MODULATION  FREQUENCIES,  HIGH  (H)  AND  LOW  (L)  INTENSITY  LEVELS. 
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ATTENTION  EFFECT  ON  VISUAL  CEPs 


TABLE  II  5.4  :  t-test  between  ATT  and  NAT  CEP  groups.  MEAN  :  average 
ATT-NAT  difference  across  N  observations,  SD  :  Standard  Deviation,  HIGH/LOW 
:  "07-24  dB  intensity  levels,  ATT/NAT  :  ATTended-to/Not-ATtended-to  CEPs,  NS 
:  Not  Significant,  P  >  |t|  of  >  0.2. 
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The  visual  CEPs  also  displayed  an  interesting  interaction  between  intensity  and  at¬ 
tention  I  Vs.  The  lower  visual  attention-related  gains  (obtained  for  low  stimulus  intensity 
level)  could  not  be  explained  by  the  limiting  noise  level  boundary.  Mean  (and  SD)  val¬ 
ues  of  noise  level  magnitudes  were  0.2  (0.1)  and  0.45  (0.2)  microvolt  RMS  at  Beta  and 
Theta  MF  regions,  respectively.  The  corresponding  values  of  absolute  CEP  magnitudes 
recorded  in  response  to  unattended-to,  low  intensity  level  stimuli  (conditions  under 
which  the  lowest  magnitudes  were  obtained),  were  higher,  0.23  (0.04)  and  0.50  (0.27) 
microvolt  RMS,  respectively  (MEAN  (SD)  values).  Thus,  more  attention-related  gain 
was  "afforded"  for  the  low  intensity  level  visual  CEP  magnitudes. 

Auditory  CEPs  did  not  display  any  significant  attention  effect  in  this  current  study, 
although  a  trend  of  higher  magnitudes  in  response  to  attended-to  stimuli  was  observed 
at  Theta  MF  region.  The  inconclusiveness  of  the  auditory  results  may  be  attributed  to 
the  well-saturated  auditory  CEP  magnitudes,  even  at  a  -12  dB  stimulus  intensity  level 
(sec  I  NT  experiment). 
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Part  III:  Bisensory  Continuous  Evoked  Potentials 


Chapter  6 

Introduction  for  The  Bisensory  Continuous  Evoked 
Potentials  Experiment 

6.1  Terminology  and  approach  comparison 

Sensory  interaction  has  been  inferred,  when  response  variations  of  one  sensory 
channel  could  be  related  to  parameter  changes  of  an  additional  stimulus,  simultaneously 
presented  to  another  sensory  channel.  This  introduction  reviews  Evoked  Potential  (EP) 
studies  in  which  the  Auditory- Visual  Interaction  (AxV)  has  been  investigated  in  humans, 
and  presents  some  of  the  terminology  used.  Most  of  the  following  literature  review  is 
based  on  Transient  Evoked  Potential  (TEP)  studies  in  which  averaged  EP  waveforms 
were  recorded  for  unimoda!  or  bimodal  transient  stimuli  (Walter,  1964;  Ciganek,  1966; 
Davis  et  al.,  1972;  Andreassi  and  Greco,  1975;  Walsh,  1979;  Lewis  and  Froning,  1981). 
An  AxV  process  was  proclaimed  whenever  the  algebraic  summation  of  the  unisensory 
EP  waveforms  (recorded  for  unimodal  stimuli)  significantly  deviated  from  the  bisensory 
EP  waveform  (recorded  for  bimodal  stimuli).  This  deduction  did  not  take  into  account 
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the  inherent  non-linearities  of  each  sensory  channel  response.  Also,  since  an  isolated 
sensory  response  to  a  bimodal  stimulus  could  not  be  directly  and  independently  re¬ 
corded,  the  summed  EP  results  could  be  misinterpreted.  Auditory,  visual  or  both  sen¬ 
sory  channel  responses  might  have  been  modified  by  the  bimodal  stimulus. 
Consequently,  both  independent  sensory  channels  (with  linearly  summed  responses)  or 
inter-related  sensory  channels  (with  inversely  related  responses)  could  generate  a  similar 
bisensory  response. 

Increment  and  decrement  of  the  bisensory  EP  magnitude  response  (relative  to  the 
algebraically-summed,  unisensory  EP  responses)  were  often  termed  facilitation  and 
'inhibition",  respectively.  Traditionally,  these  terms  were  used  to  describe  changes  of 
normal  baseline  activity  of  a  defined  source.  The  use  of  these  terms  in  AxV  TEP  studies 
was  therefore  ambiguous;  Variations  of  scalp  recorded  EPs  could  not  be  attributed  to 
a  single  sensory  channel,  and  the  choice  of  the  unisensory  response  as  the  sensory 
channel  "normal"  baseline  activity  was  questionable.  These  terms  arc  used,  however,  in 
the  following  review  in  order  to  simplify  comparisons  between  studies. 

The  present  study  investigated  the  AxV  phenomenon,  utilizing  a  phase-lock  tech¬ 
nique  in  recording  Continuous  Evoked  Potentials  (CEPs)  under  unimodal  and  bimodal 
stimulus  conditions.  A  similar  technique  was  used  by  Regan  and  Spekreijse  (1977),  in¬ 
vestigating  the  AxV  process  in  conjuncture  to  the  perception  of  auditory  and  visual 
spaces.  Substantial  differences  exist  between  the  phase-lock  (recording  of  CEPs)  and 
ensemble  averaging  (recording  of  TEPs)  approaches.  Bimodal  TEPs  are  affected  by  the 
stimulus  Inter  Modality  Interval  (IMI),  and  reflect  non-sensory-channel  specific,  AxV 
transient  processes.  In  contrast,  the  bimodal  CEPs  are  IMI-irrcievant  responses,  re¬ 
flecting  sensory  channel  specific,  AxV  sustained  processes.  The  lock-in  approach  is 
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therefore  preferable  in  situations  where  a  sensory  channel  response  is  monitored  under 
unpredictable,  cross-modal  stimulus  conditions. 

Results  entailed  in  the  following  literature  review  reflect  the  AxV  process  but  could 
not  be  confirmed  against  the  current  study  results.  Differences  in  recording  sites  and 
techniques  (ensemble  averaging  versus  phase-lock)  would  make  any  such  comparison 
invalid.  Sorting  out  the  AxV-related  results  of  the  following  TEP  studies  revealed 
varying  and  conflicting  trends.  The  major  experimental  parameters  governing  the  results 
are  assumed  to  be  the  following:  relative  unimodal  stimuli  intensities,  bimodal  stimulus 
IMI,  subject's  allocated  attention  and  EP  recording  sites. 

6.2  Auditory-visual  interaction  in  evoked  potentials 

In  comparison  to  visual  stimuli,  audio  stimuli  generally  generate  shorter  latency  TEP 
components  and  are  reacted  upon  with  faster  Reaction  Times  (RTs)  (Squires  et  al., 
1977).  rhis  might  be  the  reason  why  an  experiment  paradigm,  in  which  the  audio 
stimulus  lagged  the  cross-modal  visual  stimulus  (IMI  of  0-500  msec),  was  chosen  in  most 
of  the  bisensory  TEP  studies. 

Walter  (1964)  investigated  convergence  and  interaction  of  audio-visual  information 
in  the  human  non-specific  cortex,  utilizing  both  intracranial  and  scalp  recording  tech¬ 
niques  (TEPs  were  recorded  between  Cz-Oz  referring  to  a  grounded  right  mastoid). 
I  Mis  of  30-270  msec  between  flash  and  click  (lagging)  bimodal  stimuli  did  not  disclose 
any  AxV  effect.  Walter  concluded  that  the  sensory  projections  into  the  non-specific 
cortex  (frontal,  temporal  and  occipital  lobes)  were  "idiodromic"  (private  independent 
sensory  tracks),  and  that  any  bisensory  interaction  occurred  at  the  "superficial"  cortical 
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level.  Statistical  tests  of  the  actual  difference  between  the  bisensory  and  the  summed 
unisensory  TEPs  were  not  conducted. 

An  IMI-dependent  AxV  effect  was  described  by  Morrell  (1968).  TEPs  were  recorded 
from  C3-Cz  and  Cz-C4  (referring  to  a  grounded  forehead)  in  response  to  attended-to 
flash  and  click  bimodal  stimuli.  I  Mis  of  20-120  msec  (click  lag)  were  applied,  and 
140-240  msec  post-flash  epochs  were  analyzed.  Short  IMI  (less  than  70  msec  and  only 
at  C3-Cz  recording  site)  and  long  IMI  (longer  than  70  msec  and  recording-site- 
independent)  were  associated  with  "facilitated”  or  "inhibited"  bisensory  TEP  magnitudes, 
respectively. 

Different  results  were  obtained  by  Andreassi  and  Greco  ( 1 975).  TEPs  were  recorded 
from  02,Cz-left  mastoid  in  response  to  attended-to,  flash  and  noise  burst  bimodal, 
stimuli.  A  fixed  IMI  of  approximately  30  msec  (noise  burst  lag)  was  used  and  65-225 
msec  post-flash  magnitude  components  were  evaluated.  Their  data  showed  that  the  re¬ 
sponse  to  the  bimodal  stimuli  was  consistently  smaller  than  the  algebraically  summed 
unimodal  responses.  In  comparison,  Morrell's  results  (1968),  obtained  under  similar 
IMI  and  epoch  range,  showed  bisensory  "facilitation"  and  "inhibition"  for  magnitude 
responses  recorded  from  C3-Cz  and  Cz-C4,  respectively.  This  discrepancy  could  be  re¬ 
lated  to  the  different  recording  sites.  A  multi-origin  AxV  process  would  generate  a 
recording-site-dependent  potential  distribution. 

Bisensory  "inhibition"  of  medium-long  latency  magnitude  components  at  long  I  Mis 
(longer  than  100  msec)  was  commonly  detected.  Davis  et  ai.  (1972)  demonstrated  that 
this  "inhibition"  persisted  even  for  magnitude  components  generated  600-700  msec  after 
presentation  of  the  leading  bimodal  stimulus-component.  It  was  presumed  that  the 
isolated  sensory  channel  response  at  latencies  greater  than  600  msec  was  indistinguish- 
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able  from  the  noise  level.  TEPs  were  recorded  from  Cz-Ml  (referring  to  a  grounded 
forehead  j  in  response  to  equal  subjective-magnitude,  flash  and  tone  burst,  bimodai  or 
unimodal  stimulus  pairs.  I  Mis  of  0.5  and  5  sec  were  employed  and  100-200  msec  epochs 
of  the  magnitude  response  to  the  lagging  stimulus  were  analyzed.  Relative  magnitudes 
(of  the  short  I M I/long  I M I )  computed  for  inter-  and  cross-modality  stimulus  pair  con¬ 
ditions  were:  A|a  =  0.25,  A|v  =  0.59,  Vjv  =  0.51  and  Vja  =  0.71  (where  X|y  should  be  read 
as  the  relative  magnitude  of  the  X-modality  response,  given  that  a  y-modality  stimulus 
preceded;  A  or  a:  audio;  V  or  v:  visual).  These  results  indicated  that  the  auditory  TEPs 
were  more  susceptible  to  inhibition  and  that  the  inhibitory  effect  of  intra-modality 
stimulus  was  more  profound  (in  both  sensory  channels). 

Walsh  ( 1079)  and  Lewis  and  Frorung  (1981)  utilized  a  simultaneously-presented  (zero 
I  MI)  audio- visual  bimodai  stimulus  and  obtained  conflicting  results  regarding  the  AxV 
process.  Walsh  (1979)  recorded  TEPs  from  Cz-Oz  (referring  to  a  grounded  forehead)  in 
response  to  equal  sensation  level  magnitude  tone  burst  and  flash  bimodai  stimuli.  Walsh 
concluded  that  magnitude  components  within  the  100-150  msec  latency  epoch  did  not 
exhibit  any  AxV  phenomenon.  Lewis  and  Froning  (1981)  recorded  TEPs  from 
F3,F4,T.\  1 4,P3,P4,01,02-nosc  (referring  to  a  groundcJ  Pz)  in  response  to  click  and 
checkerboard  pattern  bimodai  stimuli.  They  found  a  recording-sitc-independent 
bisensory  inhibition"  in  250-500  msec  magnitude  components.  Shorter  latency  magni¬ 
tude  components  showed  rccording-site-dependcnt,  mixed  AxV  trends.  A  bisensory 
"facilitation  of  150  msec  magnitude  components  could  be  detected  in  TEPs  recorded 
from  frontal  recording  sites  (F3, 4  >  T3,4  >  P3,4). 

I  he  effects  of  cross-modal  audio  stimulation  on  visual  EPs  were  investigated  by 
Ciganck  .  I960)  and  Regan  and  Spckrcijse  (1977).  Ciganek  (1966)  recorded  TEPs  from 
0/-IV  u  response  to  click  and  flash  bimodai  stimuli.  !MIs  of  40-250  msec  were  em- 
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ployed  (flash  lag),  and  the  visual  unimodal  TEP  was  compared  to  the  visual  1  I  P  com¬ 
ponent  within  the  bimodal  response.  Although  significant  AxV  effects  were  not  reported 
by  Ciganek,  analysis  of  his  raw  data  revealed  some  consistent  trends.  Short  (-10-100 
msec)  and  long  (150-250  msec)  IMls  were  associated  with  a  visual  TEP  "inhibition  be¬ 
ginning  at  magnitude  components  w'ith  latencies  of  120-130  and  70-80  msec,  respectively. 

Regan  and  Spekreijse  (1977)  applied  a  phase-lock  technique  in  monitoring  the  effect 
of  cross-modal  audio  stimulation  on  visual  CEP.  The  visual  stimulus  consisted  of  a  fixed 
Modulation  Frequency  (MF  of  12,  13.7,  16  or  18  Hz),  60%  modulation  depth,  Ampli¬ 
tude  Modulated  (AM),  Maxwellian  view  of  xenon  light  source.  The  audio  stimuli  con 
sisted  of  variablc-MF,  100%  modulation  depth,  AM  sounds  (tone  bursts  or  clicks;.  The 
audio  MF  was  linearly  varied  (triangularly  frequency  modulated)  across  the  centered, 
fixed  visual  MF.  Therefore,  if  the  perception  of  "visual  flicker  rate  driven  by  the  audio 
flutter  rate"  had  an  EP  basis,  it  could  be  reflected  in  the  visual  CEP  magnitude  response. 
Visual  CEP  magnitude  variations  were  expected  whenever  the  audio  and  visual  stimuli 
MFs  were  similar,  twice  within  the  audio  MF  sweep  period.  No  such  periodic  AM  visual 
CEP  magnitude  response  was  found,  possibly  due  to  the  following  reasons:  (1)  The 
visual  CEP  magnitude  could  have  been  afTccted  evenly  during  the  entire  range  (relatively 
small)  of  the  audio  MF  sweep  range.  (2)  The  triangular  waveform  duration  was  20  sec, 
thus  similar  audio  and  visual  MFs  existed  for  only  a  short  time  (5-10  sec).  With  a  min¬ 
imal  Lock-In  Amplifier  (L1A)  output  Time  Constant  (TC)  of  3-10  sec,  magnitude  vari¬ 
ations  due  to  an  AxV  process  would  be  indistinguishable  from  noise-related  variations. 
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Chapter  7 

Methodology  for  The  Bisensory  Continuous  Evoked 

Potentials  Experiment 


7.1  Experiment  paradigm  and  variables 

Each  run  in  this  experiment  consisted  of  simultaneously  presented,  partially  overlap¬ 
ping  audio  and  visual  stimulus  scries.  The  audio  series  always  led.  Each  series  consisted 
of  three  repetitive  stimuli  (see  TABLE  I  3.3  and  FIGURE  III  8.1).  Average  stimulus 
duration  (S),  Inter  Stimulus- Interval  (ISI)  and  Audio-Visual  stimulus  Delay  (A VD)  were 
106.5,  35.5  and  71  seconds,  respectively.  Stimulus  durations  were  randomized  within  the 
range  of  -5%  to  +5%  of  the  mean  value.  Four  audio-visual  Modulation  Frequency 
(MF)  combinations  were  tested  (Beta-Beta,  Theta-Beta,  Beta-Theta,  Theta-Theta, 
presented  in  this  order),  each  at  four  audio-visual  intensity  combinations  (Low— Low, 
Low-lligh,  High-Low,  High-High,  presented  in  this  order).  The  High  and  Low  terms 
were  attributed  to  the  '0”  and  -12  dB  intensity  levels,  respectively.  Each  run,  one  of  the 
16  MF  x  intensity  combinations  lasted  for  an  average  of  7  minutes,  50  seconds.  The 
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entire  Auditory- Visual  Interaction  (AxV)  recording  session,  including  trial  runs  and  test 
breaks,  required  an  average  of  three  hours. 

The  subjects  were  instructed  to  attend  to  the  bimodal  stimuli  and  to  quickly  respond 
to  a  stimulus  tuming-ofT  clue  (Reaction  Time  test,  RT).  After  recording  each  of  the 
MF  combinations  and  before  disclosing  their  actual  RT  scores,  the  subjects  were  asked 
to  evaluate  their  performance.  They  had  to  describe  their  attention,  concentration  and 
quickness  levels  and  the  degree  of  difficulty  encountered  in  maintaining  that  level  (a  five 
point  scale  for  each  qualifier).  In  addition,  at  the  end  of  the  AxV  recording  session,  the 
subjects  were  asked  to  assess  their  inter-sensory  channel  attention  allocation  strategies. 
All  of  the  subjects  confirmed  that  during  the  bimodal  stimulus  sections,  they  attended 
to  the  modality  whose  stimuli  were  expected  to  be  turned  off. 

Audio  and  visual  stimulus  gates  (A  GATE,  V  GATE)  and  the  investigated  within-run 
Independent  Variables  (IVs)  are  presented  in  FIGURE  III  8.1.  The  analyzed  run  sec¬ 
tion  consisted  of  two  repetitions  of  the  four  basic  bimodal  combinations  (avoiding  the 
runs'  "on"  and  "off  transitions).  Statistical  analysis  was  conducted  on  the  cross-pattem 
averaged  data.  Continuous  Evoked  Potential  (CEP)  magnitudes  and  phases  were  de¬ 
fined  by  the  levels  of  the  following  IVs:  SIG,  SF,  CF,  SI,  Cl,  ORDER,  MOD,  and  ATT. 
A  summary  of  all  the  AxV  experiment  variables  is  provided  in  TABLE  III  8.1  A.  NL 
magnitudes  were  recorded  for  stimulus-off  condition  while  the  Lock-In  Amplifier  (LIA) 
was  still  locked- on  the  stimulus  MF.  Except  for  NAME  (subjects'  codes),  all  the  other 
IVs  were  treated  as  fixed  effects  in  the  tested  ANOVA  models. 
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AxV  EXPERIMENT  PARADIGM  AND  INDEPENDENT  VARIABLES 
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FIGURE  III  8.  1  :  AUDITORY  (A)  AND  VISUAL  (V)  STIMULUS  GATES 
(GATE)  AND  THE  WITHIN-RUN  INDEPENDENT 
VARIABLES.  AVERAGED  STIMULUS  ON  DURATION 
(GATE  HIGH)  WAS  106.5  SECONDS.  STIMULUS-OFF 
TRANSIENT:  REACTION-TIME  CLUE,  ATT/NAT: 
ATTENDED-/NOT-ATTENDED-TO  STIMULUS,  BI/UNI: 
BIMODAL/UNIMODAL  STIMULUS,  NL:  NOISE  LEVEL 
RESPONSE  (STIMULUS -OFF  CONDITION). 
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AxV  EXPERIMENT  INDEPENDENT  VARIABLE  DEFINITIONS 


TABLE  III  8.1  A  :  Definitions  of  the  independent  variables. 


INDEPENDENT 

CLASS 

VARIABLES 

|  ACROSS - 
IWITHIN- 
|  RUNS 

| LEVELS 
| (CLASSES) 

j 

NAME 

j  ACROSS 

1 

| AM,CR,EW, 
| KC , MR , PM , 

|RT,SC,TN 

1 

SIGnal 

| ACROSS 

1 

(Auditory, 

(Visual 

i 

SFrequency 

1  ACROSS 

1 

| Beta, 
(Theta 

1 

CFrequency 

| ACROSS 

1 

|Beta, 

| Theta 

i 

SIntensity 

| ACROSS 

| High , 

|Low 

1 

CIntensity 

(ACROSS 

1 

|High, 

I  Low 

I 

ORDER 

I WITHIN 

1 

| 1st, 2nd, 
j  3rd,NL 

1 

MODalities 

| WITHIN 

1 

|BImodal, 

juNImodal 

1 

1 

ATTention 

| WITHIN 

1 

1 

| ATTended, 
(Not-ATt- 
| ended -to 

DEFINITION 


subjects'  codes 


stimulus  modality  &  the  re¬ 
corded-  from  sensory  channel 

modulation  frequency  region 
of  the  SIGnal  stimulus 

modulation  frequency  region 
of  the  cross-SIGnal  stimulus 

I 

, 

intensity  level  of  the 
! SIGnal  stimulus 

intensity  level  of  the 
cross-SIGnal  stimulus 

I 

! SIGnal' s  response  sections 
|lst--3rd:  evoked  potentials, 
|NL:  noise  level  response 

I 

| SIGnal 's  stimulus  modalities 
|(BI  and  UNI  classes  are  the 
list  &  3rd  and  2nd  ORDER 
| sections,  respectively) 

I 

| SIGnal 's  stimulus  attention 
[allocation  (ATT  and  NAT  cla¬ 
sses  are  the  3rd  &  1st  ORDER 
| sections,  respectively) 
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7.2  Auditory-visual  interaction  assessment 


The  AxV  was  characterized  by  the  promoting  stimulus  conditions  (the  IVs)  and  the 
resultant  trend  directions  (inhibited  of  facilitated  magnitudes,  leading  or  lagging  phases). 
Consequently,  two  orthogonal  analyses  were  performed  to  answer  the  following  two 
questions: 

"Which  IVs  and  IV  interactions  induced  any  significant  AxV  trend?' 

"Which  IVs  were  significant  in  promoting  the  AxV  trend  peaks?' 

The  first  question  was  investigated  by  a  simple  ANOVA  where  the  mean  sum  of 
squares  of  each  effect  was  tested  for  significance.  The  second  question,  regarding  the 
parameter-space  origin  of  the  AxV  trend,  was  investigated  by  a  'Trend  Analysis'.  This 
analysis  was  an  iterative  process  in  which  IVs  associated  with  the  highest  F-ratio  (the 
highest  significance  level  of  the  difference  between  the  IVs  levels)  were  successively  se¬ 
lected.  Chosen  IV  levels  (enhancing  the  investigated  trend)  successively  reduced  the 
data-base  of  the  following  iteration  (a  nested  structure).  This  analysis  always  converged 
to  the  trend  peak  value.  It  provided  an  entering  priority  list  and  levels  of  the  IVs  en¬ 
hancing  that  trend.  The  contribution  of  a  lower  priority  IV  (late  to  enter  and  therefore 
deduced  from  a  smaller  sample  size)  was  regarded  as  less  reliable. 

Three  data-base  versions:  absolute",  'compensated'  and  'relative'  were  employed. 
The  absolute  data  set  was  comprised  of  the  raw  data  magnitudes  (in  dB,  relative  to  1 
microvolt  RMS)  and  phases  (in  degrees).  It  was  used  in  testing  the  NL  magnitude  de¬ 
pendency  on  the  IVs.  The  relative  data  set  was  comprised  of  the  within  run 
magnitude-gains  (in  dB)  and  phase-shifts  (in  degrees)  of  the  bimodal  CEPs  relative  to  the 
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unimodal  CEPs  (within  NAME  x  SIG  x  SF  x  SI  x  CF  x  Cl  groups).  It  was  used  to  as¬ 
sess  the  factors  contributing  to  the  AxV  phenomenon.  The  compensated  data  set  con¬ 
tained  mean-reduced  absolute  data  (absolute  value  -  group  mean  difference,  within 
NAME  x  SIG  x  SF  x  SI  groups).  This  data  set  was  used  to  expose  contributions  of 
inter-  or  cross-modality  IVs  (the  data  was  compensated  for  the  intra-modality  SF  and 
SI  effects).  The  dependent  variables  and  the  data  set  definitions  are  summarized  in 
TABLE  III  8. IB. 


AxV  EXPERIMENT  DEPENDENT  VARIABLE  DEFINITIONS 


TABLE  III  8.1B  Definitions  of  the  dependent  variables  and  the  data-bases. 


DATA-BASE 

ATTRIBUTES 


DEFINITIONS  :  Magnitude  &  Phase  VARIABLES 


ABSOLUTE 


absolute  magnitude  (dB  relative  to  1  microvolt 
RMS)  and  phase  (degrees)  values 


COMPENSATED | group -mean- reduced  absolute  data  (within  groups 
lof  NAME  x  SIG  x  SF  x  SI) 

I 

RELATIVE  | magnitude-gains  and  phase-shifts  between  MOD 
jelasses  (BImodal-UNImodal  CEP  difference) 


Exposing  the  AxV  phenomenon  within  the  auditory  and  visual  CEPs  utilized  prima¬ 
rily  the  relative  data  set.  The  following  linear  model  describes  the  AxV  contributing 
factors.  In  this  model,  B1  represents  the  1st  and  3rd  run  components,  UNI  represents 
the  2nd  run  component  (a  component  can  be  magnitude  or  phase),  and  a,b,c,d,e,a',b' 
are  weighting  coefficients  (partial  regression  coefficients).  The  symbol  stands  here 
for  multiplication. 
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BI  =  a  *  SF  +  b  *SI+c*CF+d*CI+e*  ATT  +  error 
UNI  =  a'  *  SF  +  b'  *  SI  +  error' 


"AxV"  :  BI-UNI  =  (a-a')  *  SF  +  (b-b’)  *  SI  "internal  Effects" 

+c*CF+d*CI+e*  ATT  "External  Effects" 

+  (error  -  error')  "Error  Effects" 


To  summarize,  using  the  relative  data  set,  both  "Internal"  (due  to  SF  and  SI)  and  "Ex¬ 
ternal"  (due  to  CF,  Cl  and  ATT)  lVs  might  have  contributed  to  the  AxV  phenomenon. 

No  restrictions  were  imposed  on  the  data-bases,  since  only  less  than  3%  of  the  run 
data  could  definitely  be  regarded  as  noise.  A  run  was  "noisy"  if:  the  average  CEP 
magnitude  (of  1st,  2nd  and  3rd  ORDER  sections)  was  lower  than  the  value  of  the 
group's  average  NL  +  1  Standard  Deviation  (SD)  magnitude,  and  concurrently,  the 
CEP  phase  SD  exceeded  the  group  SD  (tested  in  SIG  x  SF  groups). 

The  distribution  of  the  relative  data-base  appeared  symmetrical  (|skewness|  <  1),  and 
normality  could  be  assumed  for  the  magnitude  data.  After  verifying  the  similarity  be¬ 
tween  parametric  and  non-paramctric  tests  (due  to  the  data  distribution  symmetry), 
parametric  statistics  were  invoked  for  both  magnitude  and  phase  data.  ANOVA  and 
MANOVA  of  balanced-cells,  repeated  measure  design  saturated  models  were  tested. 
Since  IV  contributions  could  not  have  been  a-priori  hypothesized,  and  due  to  the  small 
sample  size  of  the  study,  care  must  be  exercised  in  interpreting  the  AxV  results.  There¬ 
fore,  only  highly  significant  or  integrable  resulti  were  considered  and  discussed.  Overall, 
a  rather  conservative  approach  was  employed. 
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Chapter  8 

Results  of  The  Bisensory  Continuous  Evoked 
Potentials  Experiment 


8.1  Raw  data  and  results  outline 

Auditory  and  visual  Continuous  Evoked  Potentials  (CEPs),  recorded  in  a  typical  run, 
are  demonstrated  in  FIGURE  III  8.2  .  Noise  Level  (NL)  sections  showed  the  lowest 
magnitudes  and  unstable,  random- like  phases.  Absolute  NL  magnitudes  were  checked 
for  stimulus-independent  artifacts,  biological  (EEG  contribution)  and  instrumentational 
("cross-talk"  between  similar  Modulation  Frequencies,  MFs,  due  to  a  lack  of  adequate 
Lock-In  Amplifier,  LI  A,  frequency  resolution).  Relative  CEP  data  were  tested  for 
stimulus-dependent,  physiological  Auditory-Visual  Interaction  (AxV)  effect.  The 
ANOVA  results  consist  of  tabulated  statistical  summaries  and  figures  highlighting  sig¬ 
nificant  or  important  AxV  effects.  The  trend  analysis  provided  schematics  of  the  effec¬ 
tive  Independent  Variables  (IVs)  inducing  the  peak  AxV  trends.  These  schematics  model 
the  data  structure,  but  not  necessarily  the  invoking  sensory  system  structures. 
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In  the  following  text,  and  whenever  it  was  possible,  the  analyzed  IVs  were  explicitly 
named.  However,  IV  abbreviations  and  statistical  notations  had  to  be  employed  to  fa¬ 
cilitate  comprehension  of  complex  dependencies  (efTect  interactions).  The  IVs  included 
in  an  ANOVA  model  are  termed  "effects"  (SIG,  SF,  CF,  SI,  Cl  and  ATT  class  vari¬ 
ables).  Effects  can  be  "main  effects"  (i.e.  SF  CF  )  and  "interactions'  (i.e.  SF  *  CF),  de¬ 
scribing  first-order  and  higher-order  dependencies,  respectively.  The  bar  notation,  where 
X|Y  is  short  for  X,  X  *  Y  and  Y  effects,  was  frequently  used  to  describe  the  tested 
ANOVA  models. 

8.2  Data  description  of  the  runs 

The  general  trends  of  the  compensated  runs'  magnitudes  and  phases  are  disclosed  in 
FIGURES  III  8.3,  8.4.  Magnitude- wise,  the  auditory  CEPs  revealed  a  smaller  inter- 
subject  variability  and  MF-dependent  MOD  effect  trends.  That  is,  bimodal  and 
unimodal  stimuli  were  associated  with  different  CEP  magnitudes  (FIGURE  III  8.3). 
The  visual  CEP  average  magnitude  was  clearly  influenced  by  two  subjects  (Subjects  1 
and  6),  whose  responses  to  the  audio-attended,  bimodal  stimuli  (the  1st  ORDER  mag¬ 
nitude,  see  FIGURE  III  8.1)  were  completely  suppressed.  In  general,  the  visual  re¬ 
sponse  was  associated  with  larger  absolute  CEP  magnitudes  and  lower  absolute  NL 
magnitudes.  Phase-wise,  both  modalities  were  associated  with  high  inter-subject  vari¬ 
ability  and  displayed  MF-dependent,  opposing  trends  of  progressing  phase-shifts  (pri¬ 
marily  for  the  visual  CEPs,  FIGURE  III  8.4). 

Compensated  magnitudes  of  the  runs  were  further  characterized  by  these  three  sets 
of  categorizing  IVs:  SF-CF,  SI-CI  and  CF-CI  (FIGURES  III  8.5-8.7).  The  IV  defi¬ 
nitions  arc  provided  in  TABLE  III  8.1A.  These  figures  were  also  referred  to  when  NL 
and  relative  magnitudes  were  analyzed. 
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RUN  EXAMPLE  OF  THE  AUDITORY-VISUAL  INTERACTION 

EXPERIMENT 


FIGURE  III  8.2  :  LOCK-IN  AMPLIFIER  (LIA)  VECTOR  OUTPUT  OF 

AUDITORY  (A)  AND  VISUAL  (V)  CEP*  AND  STIMULUS 
GATES  (GATE).  LIA  OUTPUT  TIME -CONSTANT:  10  SEC 
OBTAINED  FROM  SUBJECT  PM  UNDER  THETA— THETA 
MODULATION  FREQUENCY  REGIONS  AND  HIGH— HIGH 
INTENSITIES  A--V  CONDITIONS.  STIMULUS-OFF 
TRANSITION:  REACTION-TIME  CLUE. 
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AXV  CEP  MAGNITUDES 
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MAGNITUDE  SI  *  Cl  EFFECT 
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PIGURB  m  B.7  :  MEAN  COMPENSATED  MAGNITUDES  IN  CF  X  Cl  X  ORDER  GROUPS.  CP /CL 
CROSS-MODAUTr  STIMULUS  MODULATION  FREQUENCY  (MF) /INTENSITY, 
B/T:  BBTA/THKTA  MF,  H/L  "0”/-lB  DB  INTENSITY  LEVEL, 

THICE/THIN  BAR:  AUDITORY/VISUAL  DATA.  NL:  NOISE  LEVEL. 
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8.3  Noise  level  absolute  magnitudes 


The  results  of  the  ANOVA  of  auditory,  visual  and  both  combined  NL  absolute 
magnitudes  (in  dB)  are  presented  in  TABLE  III  8.2.  Since  the  N'L  magnitudes  were  re¬ 
corded  under  no  intra-modality  stimulus  condition  (see  FIGURE  III  8.1),  some  of  the 
investigated  IVs  were  attached  different  meanings.  SIG  and  SF  were  interpreted  as  the 
effects  of  the  recording  site  and  noise  frequency- band,  respectively,  and  SI  effect  (the 
intra-modality  stimulus  intensity)  became  irrelevant.  The  SI  IV  was,  however,  sepa¬ 
rately  tested  in  order  to  discard  the  possibility  of  magnitude  response  alliasing  due  to 
too-long  LI  A  TCs;  if  the  NL  magnitudes  were  predominantly  composed  of  the  pre¬ 
ceding  response  tails,  they  would  exhibit  an  SI  effect.  Since  no  such  stimulus 
intensity-NL  magnitude  dependency  could  be  demonstrated,  the  exclusion  of  the  SI  IV 
from  the  NL  magnitude  model  was  justified  (upper  part  of  TABLE  III  8.2). 

NL  magnitudes  significantly  differed  between  noise  frequency-bands  (SF  effect,  |Beta| 
<  |Theta|)  and  recording  sites  (SIG  effect,  |auditory|  >  |visual|).  The  notation  of  |IV 
class!  symbolizes  the  averaged  CEP  magnitude  at  that  IV  class.  The  actual  CEP  NL 
magnitude  ratio  values  at  each  of  the  SIG  x  SF  groups  arc  provided  in  the  legend  of 
FIGURE  III  8.3.  In  addition  to  the  SF,  SF  *  SIG  and  SIG  effects  (*  means  inter¬ 
action),  two  other  effects  were  considered.  The  significant  CF  *  Cl  effect  and  the  non¬ 
significant  SF  *  CF  effect  (TABLE  III  8.2). 

The  SF  *  CF  effect  demonstrated  the  LIA  fine  frequency  resolution.  The  overall 
trend  (predominantly  of  the  visual  CEP  recording  site)  was  of  smaller  NL  magnitudes 
associated  with  'same’  inter-modality  SF  attributes.  Beta-Beta  and  Theta- Theta  MF 
region  combinations  (similar  SF  and  CF  MF  attributes)  produced  the  smallest  magni¬ 
tudes  (FIGURE  III  8.5).  This  observation  proved  that  no  instrumentational  "cross- 
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tali."  artifacts  had  occurred.  This  conclusion  is  important,  since  an  SF  *  CF  *  ATT 
significant  effect  was  detected  for  the  visual  relative  magnitudes  (see  TABLE  III  8.5  in 
the  next  section). 

The  CF  *  Cl  effect  exclusively  resided  within  NL  magnitudes  obtained  from  the 
auditory  CEP  recording  site  (TABLE  III  8.2,  FIGURE  III  8.7).  This  effect  could  be 
interpreted  as  a  Cl-dependent  CF  effect,  where  higher  "auditory"  NL  magnitudes  (re¬ 
corded  from  Cz-Al/nose  sites)  were  associated  with  visual  Beta  MF  stimuli  (FIGURE 
III  8.7).  This  effect  was  free  of  any  intra-modality  stimulus  parameter  influence  (no 
significant  SF  *  CF  *  Cl,  SI  *  CF  4  Cl  or  SF  *  SI  *  CF  *  Cl  effects). 

8.4  Bimodal  -  unimodal  relative  auditory  CEPs 

8.4.1  Analysis  of  variance 

The  results  of  the  ANOVA  of  relative  CEP  magnitudes  and  phases,  and  the 
MANOVA  of  both  together  are  presented  in  TABLE  III  8.3.  These  relative  values  (the 
bimodal  -  unimodal  CEP  magnitudes  and  phases  differences)  reflect  the  AxV  phenom¬ 
enon  (as  defined  in  section  III  7.2).  Cross-subject  averages  of  relative  magnitudes  and 
phases  were  computed  for  the  32  possible  SF  x  SI  x  CF  x  Cl  x  ATT  combinations  (five, 
two-levels  IVs),  and  their  univariate  tests  are  provided  in  TABLE  III  8.4.  Relative 
magnitudes  and  phases  were  clearly  unrelated.  None  of  the  model's  tested  effects  ex¬ 
hibited  an  increase  of  the  MANOVA  significance  level  score  over  that  of  the  ANOVA 
scores  (TABLE  III  8.3).  Therefore,  relative  magnitudes  and  phases  were  separately  an¬ 
alyzed. 
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AxV  NOISE  LEVEL  MAGNITUDES 


TABLE  III  8.2  :  ANOVA  and  MANOVA  of  repeated- measure  models.  M:  absolute 
noise  level  Magnitudes  (in  dB).  Notation:  X|Y  is  equivalent  to  X  X*Y  Y,  MS: 
Mean  Square  (using  Type  III  sum  of  squares),  DF:  Degrees  of  Freedom  (num, 'den), 
PR>  F  significance  level  codes:  .  >  0.1,  a  <  0.1,  b  <  0.05,  c  <  0.01,  d  <  0.005, 
e  <  0.001,  f  <  0.0005,  g  <  =  0.0001. 


■EFFECTS 


Tested  against 


ANOVA  OF  NOISE  LEVEL  MAGNITUDES 
F-tests,  1/8  DF 


NAME*ef fects  |  AUDITORY  (A)|  VISUAL  (V)  |  A  &  V 
error  terms  j  MS  (F,  PR>F  j  MS  |F,  PR>F  |  MS  [F,  PR>F 

NAME  |  837 |  |1024|  | 1658 | 

SF  ~  j 2355 |91.1  g  | 1101 | 132  g  | 3339 | 184  g 

CF  1 19. 3 1  4. 6  a  Jll.  7  j  2.6  .  |  0.5 1  0.1  . 

SF*CF  j  0.9 1  0.1  .  |29.8 j  1.8  .  1 10.  1|  1.4  . 

Cl  |  1.9 1  0.2  .  |  0.0 1  0.0  .  j  1.2 1  0.4  . 

SF*CI  |  6. 0 1  1. 6  .  |  0. 2|  0.0  .  |  4.  1|  0. 3  . 

CF*CI  1 23. 5 1  8.  5  b  |  0.  1|  0.0  .  1 10-  5 1  1.1  . 

SF*CF*CI  |  0. 4|  0.  1  .  |  2. 2|  0. 3  .  |  0.4|  0. 0  . 

SIG . 1-  -  | - |  442(17.5  d 

SIG*SF . |-  -  | - I-  -  I - I  H8|  7.3  b 

SIG*CF . j-  -  j - j-  -  j - 1 30.  5  j  6.  2  b 

SIG*SF*CF - |-  -  | - j-  -  j - 1 20.  6 1  1.3  . 

SIG*CI . |-  -  j- - j—  —  j-  -  -  —  |  0. 8|  0. 0  . 

SIG*SF*CI  -  -  -|-  -  j- - j-  -  | - |  2.  1 1  0. 2  . 

SIG*CF*CI  -  -  -j-  -  | - j-  -  j- - |13.1 1  1.0  . 

S IG*SF*CF*C I -  -j-  -  I - 3  0.4  . 


MODEL:  M=  SF|CF|CI |NAME,  for  A  and  V  separate  data  sets 
MODEL:  M=  SIG|SF|CF|CI (NAME,  for  A  &  V  combined  data  set 


SI  |  0.  1|  0.0  .  |  0. 4 1  0.0  .  j  0. 4 1  0.0 

SF*SI  j  6.6  j  1.4  .  1  7.3  j  0.7  .  |  0.0 1  0.0 

SIG*SI . I--I- - I--I - |  0.  0|  0.0 

SIG*SF*SI - 1-  -  | - -  I-  -  | - |13.9 1  2.5 


MODEL:  Ms  SF|SI|NAME,  for  A  and  V  separate  data  sets 
MODEL:  M=  SIG| SF | SI | NAME ,  for  A  &  V  combined  data  set 
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8.4. 1.1  A  uditory  relati ve  magnitudes 


Auditory  relative  magnitudes  were  primarily  affected  by  the  audio  stimulus  MF  (SF 
effect)  and  the  inter-modality  intensity  attributes  (SI  *  Cl  effect).  Regarding  the  SF  ef¬ 
fect,  Beta  and  Theta  auditory  MF  regions  were  associated  with  decrease  (bisensory 
"inhibition")  and  increase  (bisensory  "facilitation")  of  CEP  relative  magnitudes,  respec¬ 
tively  (FIGURE  III  8.3,  TABLE  III  8.4).  This  effect  was  independent  of  the  visual 
stimulus  MF,  inter-modality  intensities  and  attention. 

Facilitated  relative  magnitudes  (|BI|  >  |UNI|)  were  also  associated  with  "different" 
inter-modality  intensity  attributes  .  Larger  relative  magnitudes  w'ere  associated  with 
High— Low  and  Low— High  Audio-Visual  stimulus  intensity  conditions,  further  resolved 
by  attention  (SI  *  Cl  and  SI  *  Cl  *  ATT  effects,  TABLE  III  8.3  and  FIGURE  III  8.8). 
This  SI  *  Cl  effect  could  be  interpreted  as  an  audio  stimulus  intensity-dependent,  visual 
stimulus  intensity  effect  (Cl  effect  of  |Low|  >  | High]  at  SI  =  High,  FIGURE  III  8.8). 

8. 4. 1.2  Auditory  relative  phases 

Auditory  relative  phases  were  primarily  affected  by  the  SI  *  CF  *  ATT  interaction 
(TABLE  III  8.3,  FIGURE  III  8.9).  FIGURE  III  8.9  reveals  that  the  SI  *  CF  effect 
within  the  "MAT"  attention  level  was  the  source  of  that  phenomenon.  The  combinations 
of  Beta  visual  stimulus  MF-High  audio  stimulus  intensity  and  Theta  visual  stimulus 
MF— Low  audio  stimulus  intensity  were  associated  with  leading  relative  phases  (BI  > 
UNI  at  Beta  CF-High  SI  and  Theta  CF-Low  SI  combinations,  FIGURE  III  8.9). 
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RELATIVE  (BIMODAL-UNIMODAL)  AUDITORY  CEP  VECTORS 


TABLE  III  8.3  :  ANOVA  and  MANOVA  of  repeated-measure  models,  M:  relative 
CEP  Magnitudes  (in  dB)  ,  P:  relative  CEP  Phases  (in  degrees).  Notation:  XjY  is 
equivalent  to  X  X*Y  Y,  MS:  Mean  Square  (using  Type  III  sum  of  squares).  DF: 
Degrees  of  Freedom  (num/den),  PR>F  significance  level  codes:  .  >  0.1,  a  <  0.1, 
b  <  0.05,  c  <  0.01,  d  <  0.005. 


- EFFECTS - 1 . AN 

Tested  against |  F-tests, 

NAME*ef fects  j MAGNITUDES  (M) 

error  terms  j  MS  |F,  PR>F 

1  1 

OVA . . 

DF:  1/8 
|  PHASES  (P) 

j  MS  1 F,  PR>F 

1  1 

| -MANOVA - 
IDF:  2/7 

1  M  P 
IF,  PR>F 

1 

1 

1 

1 

NAME 

SF 

-1  1 

1 164.  2 j 
|277.  4| 26. 7 

e 

I 

1 

1 

l 

1049| 
84 1 

0.  1  . 

1 

111.7  c 

1 

! 

SI 

j  5.  5  j  0.6 

1 

852  j 

0. 9  . 

j  1.  1  . 

1 

SF*SI 

|  11.9|  1.5 

1 

1878| 

1.4  . 

j  1.  1  . 

1 

CF 

j  4.  7  j  0.4 

1 

32  j 

0. 0  . 

|  0.  3  . 

1 

SF*CF 

|  24.  1|  1.2 

1 

HI 

0.  0  . 

j  0. 6  . 

1 

SI*CF 

j  10.  1  j  1.  2 

1 

6569| 

2.4  . 

j  1.  8  . 

1 

SF*SI*CF 

j  11.  1 1  0.  6 

1 

4| 

0. 0  . 

|  0.  3  . 

1 

Cl 

|  46.  4 1  2.3 

1 

8618| 

2. 5  . 

|  4.0  a 

1 

SF*CI 

|  20.  0  j  1.  1 

1 

243| 

0.  1  . 

1  0.  7  . 

1 

SI*CI 

j  26.0116.4 

d 

1 

4| 

0. 0  . 

1 10.  6  c 

1 

SF*SI*CI 

|  0.  5|  0.  0 

1 

2308| 

0.  7  . 

|  0.  3  . 

1 

CF*CI 

|  20.  9 1  1.3 

1 

175| 

0.0  . 

1  0.6  . 

1 

SF*CF*CI 

|  3.  5  |  0.2 

1 

9287| 

3. 0  . 

j  1.3  . 

1 

SI*CF*CI 

j  0.  4  j  0.0 

1 

263| 

0.  1  . 

j  0.  0  . 

1 

SF*SI*CF*CI 

|  13.  3 1  0.4 

1 

3169| 

1.  1  . 

|  0.5  „ 

1 

ATT 

|  14.0|  1.7 

1 

489 1 

0.6  . 

|  2.  2  . 

1 

SF*ATT 

|  2.  oj  0.  3 

1 

461  j 

1. 0  . 

|  0.4  . 

1 

SI*ATT 

|  8.  5  |  3.4 

1 

4| 

0. 0  . 

j  1.  8  . 

1 

SF*SI*ATT 

j  1.0|  0.2 

1 

1071| 

2.  1  . 

j  1.  1  . 

1 

CF*ATT 

|  0.  5  |  0.  1 

1 

83( 

0.  1  . 

j  0.  2  . 

1 

SF*CF*ATT 

|  6.  0 1  1.2 

1 

12691 

1. 6  . 

j  1. 0  . 

1 

SI*CF*ATT 

|  2.  4 1  1.0 

1 

3934| 

15.  3  d 

1  6.  9  b 

1 

SF*SI*CF*ATT 

|  0.6)  0.  1 

1 

7 

0. 0  . 

j  0. 0  . 

1 

CI*ATT 

|  3.  7 j  0.  8 

1 

54  j 

0. 0  . 

I  0. 4  . 

1 

SF*CI*ATT 

|  4.6j  0.5 

1 

7) 

0. 0  . 

|  0.  3  . 

1 

SI*CI*ATT 

|  7.9)  5.2 

a 

1 

346 1 

0.4  . 

|  2.  3  . 

1 

SF*SI*CI*ATT 

j  2.5!  0.6 

1 

lj 

0. 0  . 

|  0. 5  . 

1 

CF*CI*ATT 

|  0.  4  j  0.  1 

1 

303) 

0. 5  . 

I  0. 3  . 

1 

SF*CF*C I * ATT 

j  0.  3 1  0.2 

1 

20  j 

0. 0  . 

j  0.  1  . 

1 

SI*CF*CI*ATT 

j  1.  5 1  0.4 

1 

3038| 

2. 5  . 

|  1. 9  . 

1 

SF*SI*CF*CI*ATT|  6.0|  1.4 

1 

626| 

1.5  . 

j  1. 0  . 

1 

MODEL:  M  or  P  or  M  P  =  SF| SI jCF|CI | ATT|NAME 

1 
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AUDITORY  AxV  UNIVARIATE  STATISTICS 


TABLE  III  8.4  :  AXOVA  of  the  intercept  (null  hypothesis  of  M  or  P  =  0)  for  CEP 
relative  Magnitudes  (M)  and  Phases  (P).  "0"/- 1 2  dB  intensity  level,  PR>  F 

significance  level  codes:  .  >  0.1,  a  <  0.1,  b  <  0.05,  c  <  0.01. 


INDEPENDENT  VARIABLE 

SF  |  SI  |  CF  1  Cl  | 

.  i  i  i  i 

LEVELS 

ATT 

|  CEP  MEANS  | 

|  M  |P  | 

|  dB  |  deg. | 

F- 

M 

•tests 

PR>F 

!  p 

BETA 

"H" 

i  i 

(BETA  | 

"H"  | 

ATT 

-|  i 

|  -0.  78  | 

r 

7.  2 1 

1 

BETA 

"H" 

| BETA  1 

"H"  | 

NAT 

|-1.87| 

12.  2  | 

a 

1 

BETA 

"H" 

j  BETA  | 

"L"  | 

ATT 

|  0.90| 

7.  2  j 

1 

BETA 

"H" 

| BETA  j 

"L"  | 

NAT 

j  O.Slj 

3.  3  j 
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| THETA  j 
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|-1.33| 
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"H” 
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j  o.ooi 

3.  3 1 

1 

BETA 
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a 

1 

b 
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MODEL: 

M  or  P  =  intercept 

(at 

each  SF  x  SI  x  CF  x 

Cl  x  ATT  group) 
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MAGNITUDE  SI  *  Cl  *  ATT  EFFECT 
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RELATIVE  MAGNITUDE  TYPE 


FIGURE  IQ  B.8  :  MEAN  RELATIVE  MAGNITUDES  (BD40DAL-UNIMDDAL  DIFFERENCE  IN  DB). 
m/CL  8TDCUU7B/CR08S-M0DALITY  STIMULUS  INTENSITY,  H/L 

"tr/-is  ob  ORiNsirr  level,  tbick/tbjh  bar-  auditory/yisual 

DATA,  AIT/NAT/ALL  ATTSNDED/NON  -ATTEND  ED/BOTH -AVERAGED  DATA. 
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PHASE  SI  *  CF  *  ATT  EFFECT 


RELATIVE  PHASE  TYPE 


FIGURE  m  8.0  :  MEAN  RELATIVE  PHASES  (BIMODAL-UNIMOUAL  DIFFERENCE  IN 

DEGREES).  Sl/CF:  STIMULUS  INTENSITY/CROSS-MODALITY  STIMULUS 
MODULATION  FREQUENCY  (MF),  H/L;  "0"/-12  DB  INTENSITY  LEVEL, 
BA=  BETA/THETA  MF,  THICX'  'THIN  BAR:  AUDITORY/VISUAL  DATA, 
ATT/NAT/ALL:  ATTEND  ED /NCR- ATTEND  ED /BOTH -AVERAGED  DATA. 


8.4.2  Trend  analysis 


Parameter-space  origins  of  the  positive  ('  +  ')  and  negative  ("-")  relative  (BI-UNI) 
Magnitudes  (in  dB)  and  Phases  (in  degrees)  were  disclosed,  and  the  resultant  trend  paths 
are  described  in  FIGURE  III  8.10.  The  investigated  trend  in  each  path  (+  M,  -M,  +  P, 
and  -P)  was  defined  by  a  primary  (1st  priority)  stimulus  parameter,  an  IV  (one  of  the 
MFs,  INTensities  and  ATTention  IVs)  associated  with  the  highest  significance  level  in 
exhibiting  that  trend.  The  other  parameter  levels  were  successively  selected  to  enhance 
that  trend.  Each  parameter  was  selected  from  the  formerly-defined  parameter-space. 
The  values  of  the  4th  priority  parameter  levels  (averaged  across  the  levels  of  the  lowest 
priority  parameter)  made  up  the  final  trend  peak  values.  When  checked  against  the 
univariate  means  of  TABLE  III  8.4,  it  was  evident  that  this  iterative  procedure  con¬ 
verged  to  (or  conversely  was  influenced  by)  the  peak  values. 

Overall,  magnitude  trend  paths  were  resolved  with  higher  parameter  significance  lev¬ 
els.  Thus,  within  the  same  five-dimensional  parameter-space,  inhibition  and  facilitation 
trends  of  relative  magnitudes  were  significantly  more  confined  (and  consequently  more 
susceptible  to  "noisy"  data  influence).  In  comparison,  the  leading  and  lagging  trends  of 
relative  phases  were  more  general  (broader)  and  less  significant. 

Magnitude  and  phase  trends  appeared  inverted  between  the  audio  MF  regions. 
Positive  and  negative  trends  were  primarily  associated  with  the  Theta  (late  CEPs)  and 
Beta  (early  CEPs),  respectively.  Peak  trends  of  relative  magnitudes  and  phases  were 
defined  by  different  primary  parameters  and  generally  characterized  by  different  priority 
and  parameter  level  lists.  Therefore  magnitude  and  phase  trends  were  separately  as¬ 
sessed  in  the  following  paragraphs. 
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8.4.2. 1  Auditory  magnitude  trends 


The  parameter  lists  of  negative  ('-M',  bisensory  'inhibition')  and  positive  ("  +  M", 
bisensory  'facilitation')  relative  magnitude  trends  complemented  each  other.  That  is, 
all  the  parameters'  levels  deferred  between  the  trends.  Primarily  determined  by  the  Au¬ 
dio  stimulus  MF  parameter,  inhibition  and  facilitation  trends  were  associated  with  the 
Beta  and  Theta  MF  regions,  respectively.  Within  these  MF  regions,  inhibition  and  fa¬ 
cilitation  were  further  enhanced  by  High,  Audio  and  Visual  stimulus  INTcnsities,  re¬ 
spectively  (2nd  priority  parameter). 

Investigation  of  the  5th  priority  parameters  revealed  that  inhibition  and  facilitation 
enhanced  trends  were  independent  of  the  Audio  INTensity  and  the  Visual  MF,  respec¬ 
tively.  It  is  interesting  to  note  that  although  ATT  effect  was  generally  not  significant 
(see  previous  ANOVA  of  relative  magnitudes,  TABLE  III  8.3),  ATTending  to  the  Audio 
stimulus  increased  the  relative  magnitude  faciiitory  trend  and  vice  versa. 

8. 4. 2. 2  Auditory  phase  trends 

Overall,  the  two  phase  trend  paths  were  non-compiementary  and  differed  structurally. 
Positive  relative  phase  trend  ('^P',  'leading")  was  primarily  defined  by  the  Visual 
stimulus  parameters  (V  INTensity,  V  MF)  and  enhanced  by  the  Audio  stimulus  param¬ 
eters  (A  M  F,  A  INTensity).  Negative  relative  phase  trend  ("-P",  'lagging')  was  primarily 
defined  by  the  INTensity  parameters  (V  INTensity,  A  INTensity)  and  enhanced  by  the 
MF  parameters  (V  MF,  A  MF). 
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STIMULUS  PARAMETERS  INDUCING  THE  AUDITORY  PEAK  AxV 

TRENDS 
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FIGURE  III  8.10  :  TREND  ANALYSIS  OF  RELATIVE  (BISENSORY-UNISENSORY) 
CEP  MAGNITUDES  (M)  AND  PHASES  (P).  EACH  TREND¬ 
ENHANCING  PATH  WAS  AFFECTED  BY  FIVE  STIMULUS 
PARAMETERS  AND  DEFINED  BY  (LEFT  TO  RIGHT): 
PARAMETER  LEVELS,  PRIORITIES  AND  SIGNIFICANCES. 
KEY -CHARACTERS:  AUDIO  (A),  VISUAL  (V),  MODULATION 
FREQUENCY  (MF),  INTENSITY  ( INT) ,  BETA  (B),  THETA 
(T),  HIGH  (H),  LOW  (L),  ATTENTION  (ATT),  "+7"-": 
POSITIVE/NEGATIVE  BISENSORY-UNISENSORY  TREND. 

PR>F  SIGNIFICANCE  LEVEL  CODES:  .  >0.5, 
a  <  0.5,  b  <  0.  1,  c  <  0.  05,  d  <  0.01, 
a  <  0.005,  f  <  0.001. 
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Primarily  determined  by  the  Visual  stimulus  INTensity  parameter,  leading  and  lag¬ 
ging  relative  phase  trends  were  associated  with  High  and  Low  INTensity  levels,  respec¬ 
tively.  Note  that  both  phase  trends  were  ATTention- independent  and  a  clear-cut 
association  between  Auditory  and  Visual  MFs  was  not  established. 

8.5  Bimodal  -  uni  modal  relative  visual  CEPs 

8.5.1  Analysis  of  variance 

The  results  of  the  ANOVA  of  relative  CEP  magnitudes  and  phases,  and  the 
MANOVA  of  both,  are  presented  in  TABLE  III  8.5.  Univariate  means  and  tests  of 
relative  magnitudes  and  phases  are  provided  in  TABLE  III  8.6.  Similar  to  the  auditory 
relative  CEPs,  the  visual  relative  magnitude  and  phase  data  were  found  to  be  unrelated 
and  thus  were  analyzed  separately. 

8. 5. 1.1  Visual  relative  magnitudes 

Visual  relative  magnitudes  were  primarily  affected  by  the  SF  *  CF  *  ATT  and  SF  * 
SI  *  CF  *  Cl  interactions.  The  SF  *  CF  *  ATT  significant  interaction  could  be  inter¬ 
preted  as  an  attention-dependent,  inter-modality  stimulus  MF  attributes  clTcct 
(ATT/NAT-dependent  SF  *  CF  effect,  FIGURE  III  8.11).  Overlooking  the  within-run 
average  trend  of  lower  1st  magnitude  (ORDER'S  1st,  see  section  8.2),  larger  and  smaller 
relative  magnitudes  were  associated  with  ATT-" same"  inter-modality  MF  attributes  and 
NAT- "same"  inter-modality  MF  attributes  stimulus  conditions,  respectively. 
Attentiveness  conditions  are  referred  to  the  visual  stimulus. 
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RELATIVE  (BIMODAL-UNIMODAL)  VISUAL  CEP  VECTORS 


TABLE  III  8.5  :  ANOVA  and  MANOVA  of  repeated-measure  models,  M:  relative 
CEP  Magnitudes  (in  dB)  ,  P:  relative  CEP  Phases  (in  degrees).  Notation:  X|Y  is 
equivalent  to  X  X*Y  Y,  MS:  Mean  Square  (using  Type  111  sum  of  squares),  DF: 
Degrees  of  Freedom  (num/den),  PR>  F  significance  level  codes:  .  >  0.1,  a  ^  0  1, 
b  <  0.05,  c  <  0.01,  d  <  0.005. 
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VISUAL  AxV  UNIVARIATE  STATISTICS 


TABLE  III  8.6  :  ANOVA  of  the  intercept  (null  hypothesis  of  M  or  P  =  0)  for  CEP 
relative  Magnitudes  (M)  and  Phases  (P).  "0*/- 1 2  dB  intensity  level,  PR>  F 

significance  level  codes:  .  >  0.1,  a  <  0.1,  b  <  0.05,  c  <  0.01. 
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MAGNITUDE  SF  *  CF  *  ATT  EFFECT 


The  significant  SF  *  SI  *  CF  *  Cl  interaction  was  interpreted  as  an  SI  * 
Cl-dependent  SF  *  CF  effect.  The  following  relative  magnitude  means  were  calculated 
for  the  'same  MF'  and  "different  MF'  SF  *  CF  combinations  under  the  'same  INT"  and 
'different  INT"  SI  *  Cl  conditions.  'Same  MF—'same  INT",  'same  MF'— "different 
INT",  "different  MF'-'same  INT"  and  "different  M F"-'difrerent  INT"  combination 
values  were  -1.6,  -0.6,  0.4  and  -1.5,  respectively.  These  means  support  the  above  inter¬ 
pretation;  "Same  INT'  condition  enhanced  the  "same  MF"  -  'different  MF"  difference. 

8. 5. 1.2  Visual  relative  phases 

Visual  relative  phases  were  significantly  affected  by  the  SF  *  ATT  effect  (TABLE  III 
8.5,  FIGURE  III  8.4).  FIGURE  III  8.4  shows  that  non-attended  Beta  and  Theta  MF 
visual  stimuli  were  associated  with  lagging  (B1  <  UNI)  and  leading  (BI  >  UNI)  phases, 
respectively.  Attentiveness  influenced  only  the  Theta  MF  visual  responses. 

8.5.2  Trend  analysis 

Visual  trend  analysis,  similar  to  the  one  conducted  for  the  auditory  CEPs  is  presented 
in  FIGURE  III  8.12.  The  convcrged-upon  final  trend  values  matched  the  peak  values 
of  the  univariate  means  of  TABLE  III  8.6.  The  visual  trend  analysis  was  carried  out  on 
the  entire  sample  size,  including  Subjects  1  and  6.  Since  the  F-test  (which  determines  the 
parameter  selection  priorities)  is  less  susceptible  to  'outliers'  than  the  means,  the  over¬ 
riding  magnitude  contribution  of  Subjects  1  and  6  (FIGURE  III  8.3)  was  moderated. 
And  indeed,  the  parameter  priority  lists  of  the  relative  magnitude  trends  (FIGURE  III 
8.12)  incorporated  the  ATTention  parameter,  only  as  the  3rd  parameter  on  the  priority 
list.  Nevertheless,  the  magnitude  trend  paths  were  regarded  as  less  reliable  than  the 
phase  trend  paths. 
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Overall,  the  visual  trend  analysis  picture  was  not  as  sharp  as  that  of  the  auditory. 
Specific  auditory  <-->  visual  paths  (MF  region  association)  could  not  be  established. 
Contrary  to  the  auditory  paths,  all  the  visual  trends  were  associated  with  only  one 
cross-modality  MF,  the  Auditory  Beta  MF  region. 

Magnitude  and  phase  trend  paths  were  resolved  with  similar  parameter  significance 
levels.  They  were,  however,  defined  by  different  primary  parameters,  and  generally 
characterized  by  different  priority  and  parameter  level  lists.  The  following  trend  assess¬ 
ment  was  therefore  conducted  separately  for  the  magnitude  and  phase  data. 

8.S.2.1  Visual  magnitude  trends 

Both  magnitude  trends  ('  +  M',  "-M')  were  associated  with  Beta  Auditory  <-> 
Beta  Visual  MF  regions  path.  Confirming  the  relative  magnitude  trend  previously  dis¬ 
played  in  FIGURE  III  8.3,  inhibitory  trends  were  more  prominent  than  the  facilitory 
trends  (FIGURE  III  8.12).  Primarily  determined  by  the  Visual  stimulus  INTensity, 
inhibition  and  facilitation  magnitude  trends  were  associated  with  High  and  Low 
INTensity  levels,  respectively.  Both  trend  peaks  were  independent  of  the  Audio  stimulus 
INTensity  level. 

8J.2.2  Visual  phase  trends 

Primarily  determined  by  the  Audio  stimulus  INTensity,  lagging  and  leading  phase 
trends  were  associated  with  Low  and  High  INTensity  levels,  respectively.  Within  these 
INTensity  levels,  the  trends  were  further  enhanced  by  Beta  and  Theta  Visual  stimulus 
MFs,  respectively.  Both  trend  peaks  were  independent  of  the  Audio  stimulus  MF. 
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STIMULUS  PARAMETERS  INDUCING  THE  VISUAL  PEAK  AxV 

TRENDS 
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AFFECTING  AUDIO  STIMULUS 


FIGURE  III  8.12  :  TREND  ANALYSIS  OF  RELATIVE  (BISENSORY-UNISENSORY) 
CEP  MAGNITUDES  (M)  AND  PHASES  (P).  EACH  TREND¬ 
ENHANCING  PATH  WAS  AFFECTED  BY  FIVE  STIMULUS 
PARAMETERS  AND  DEFINED  BY  (LETT  TO  RIGHT): 
PARAMETER  LEVELS,  PRIORITIES  AND  SIGNIFICANCES. 
KEY-CHARACTERS:  AUDIO  (A),  VISUAL  (V),  MODULATION 
FREQUENCY  (MF),  INTENSITY  (INT),  BETA  (B),  THETA 
(T),  HIGH  (H),  LOW  (L),  ATTENTION  (ATT), 
POSITIVE/NEGATIVE  BISENSORY-UNISENSORY  TREND. 

PR>F  SIGNIFICANCE  LEVEL  CODES:  .  >0.5, 
a  <  0.5,  b  <  0.1,  c  <  0.05,  d  <  0.01, 
e  <  0. 005,  f  <  0. 001. 
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Chapter  9 

Discussion  of  The  Bisensory  Evoked  Potentials 

Experiment 


9.1  Data  validity  and  reliability 

The  basic  experimental  unit  of  the  Auditory-Visual  Interaction  (AxV)  experiment  was 
an  absolute  response  vector  (magnitude  and  phase  of  response  obtained  from  the 
auditory  or  visual  recording  sites)  of  a  single  ORDER  section  (1st,  2nd,  3rd  or  NL 
sections,  defined  in  TABLE  III  8.1A).  For  the  AxV  data  to  be  valid,  the  experimental 
units  had  to  be  "instrumentation-independent",  intra-modality  (between  sequential  OR¬ 
DER  sections)  and  inter-modality  (between  temporally-corresponding  ORDER 
sections).  The  Lock-In  Amplifier  (LIA,  PAR  EG&G  MOD:  5204)  used  in  this  study 
satisfied  both  requirements.  With  an  averaged  ORDER  section  duration  of  35.6  sec  and 
output  Time-Constant  (TC)  of  10  sec,  the  LIA  provided  adequate  temporal  resolution 
and  excellent  frequency  specificity.  As  indicated  by  the  Noise  Level  (NL)  magnitude 
results,  the  Continuous  Evoked  Potentials  (CEPs)  recorded  in  one  modality  were  not 
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"instrumentationally"  affected  by  intra-  or  inter-modality  stimulus  parameters  (no  sig¬ 
nificant  SI  *  SIG,  SI  *  SF,  SI  *  SIG  *  SF  and  SF  *  CF  effects  TABLE  III  8.2). 

In  analyzing  the  AxV  results,  it  was  assumed  that  the  within-run  CEP  variations  were 
exclusively  invoked  by  the  bimodal-unimodal-bimodal  stimulation  paradigm.  This  rea¬ 
sonable  assumption  could  not  be  directly  confirmed  or  reputed  in  this  study.  In  a  pre¬ 
viously  conducted  pilot  study,  long  records  of  unimodal  auditory  CEPs  (longer  that  90 
sec)  never  revealed  consistent  magnitude  trends,  such  as  those  described  in  FIGURE  III 
8.3.  In  any  event,  such  a  mysterious,  unimodal  significant  trend,  would  have  been  much 
harder  to  explain  without  resorting  to  the  AxV  phenomenon. 

9.2  Auditory  and  visual  bisensory  responses 

9.2.1  Trend  interpretation 

Testing  the  effects  of  a  two-level  Independent  Variable  (IV,  X  and  Y  levels),  it  was 
impossible  to  determine  whether,  for  example,  X  induced  a  positive  trend,  or  Y  (or  the 
absence  of  X)  induced  a  negative  trend,  or  both.  Without  having  a  baseline  reference, 
the  ANOVA  and  trend  analysis  model  the  data  and  not  necessarily  the  true  underlying 
invoking  system.  For  example,  consider  the  auditory  relative  magnitude  trends  (FIG¬ 
URE  III  8.10)  and  their  associated  MF  region  latencies  and  delays  (TABLE  II  4.2). 
An  "inverting"  sensory  stage  between  the  two  auditory  MF  regions'  loci  could  have  ac¬ 
counted  for  the  obseived,  MF-dependent,  sign  inversion  of  the  relative  magnitude 
trends. 

In  any  case,  the  ranges  between  the  values  of  the  positive  and  negative  trend  peaks 
indicated  a  solid  AxV  phenomenon.  Within  the  investigated  parameter-space,  average 
ranges  of  5.5  dB  (relative  magnitudes)  and  40  degrees  (relative  phases)  were  obtained. 
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9.2.2  Trend  detectability 


Trend  detectability  in  both  ANOVA  and  trend  analysis  paradigms  was  a  function  of 
CEP  Signal/Noise  Ratio  (SNR)  and  inter-  and  intra-subject  variabilities.  In  comparison 
to  the  auditory,  the  visual  response  was  characterized  by  higher  CEP  and  lower  NL  ab¬ 
solute  magnitudes  (FIGURE  III  8.3,  legend),  higher  inter-subject  variability  (compen¬ 
sated  and  relative  magnitudes  and  phases  ,  FIGURES  Ill  8.3,  8.4,  TABLES  III  8.3,  8.5, 
NAME  effect)  and  lower  intra-subject  magnitude  variability  (DISC  experiment,  FIG¬ 
URE  II  5.2).  The  findings  of  this  experiment,  regarding  the  CEP  SNR  and  inter-subject 
variability,  matched,  those  of  the  MTF  experiment  (section  4.4.2  and  TABLE  II  4.1). 

All  the  above  "features"  should  have,  retrospectively,  made  the  visual  CEP  a  "pre¬ 
ferred"  vehicle  for  the  AxV  phenomenon.  Instead,  mixed  relative  magnitude  and  phase 
trends  were  obtained  (FIGURES  III  8.10,  8.12).  Phase-wise,  visual  paths  were  slightly 
more  significant,  but  magnitude-wise,  the  auditory  paths  were  much  morc.significant  and 
reliable.  These  results  may  indicate  that  the  auditory  magnitudes  were  more  susceptible 
to  cross-modal  influence,  and/or,  the  auditory  recording  site  was  closer  to,  or  better 
oriented  towards  the  AxV  loci. 

Some  previous  evidence  has  indicated  that  the  AxV  process  was  indeed  best  recorded 
from  frontal  and  parietal,  non-specific  cortical  areas  (Walter,  1964;  Lewis  and  Froning, 
1981).  Walter  (1964)  investigated  the  convergence  of  auditory,  visual  and  tactile  re¬ 
sponses  in  the  human  non-specific  cortex,  recorded  both  from  implanted  microelectrodes 
and  surface  scalp  electrodes.  He  found  that  signals  in  all  studied  modalities  converged 
at  the  frontal  cortex  and  were  widely  dispersed  therein.  Lewis  and  Froning  (1981)  re¬ 
corded  TEPs  from  F3,F4,T3,T4,P3,P4,01,02-nose/Pz  in  response  to  click  and  checker¬ 
board  pattern  bimodal  stimuli.  They  found  a  recording-site-dependent  bisensory 


'facilitation'  in  150  msec  latency  magnitude  components,  maximized  in  TEPs  recorded 
from  more  frontal  recording  sites  (F3,4  >  T3,4  >  P3,4). 

9.2.3  Magnitude  and  phase  data  comparison 

The  results  of  the  A  NOVA  and  trend  analysis  of  both  auditory  and  visual  relative 
CEP  data  revealed  that  magnitude  and  phase  information  did  not  constructively  com¬ 
plement  each  other  (TABLE  III  8.3,  8.5,  FIGURES  III  8.10,  8.12).  Within  the  investi¬ 
gated  parameter-space,  magnitude  and  phase  trends  differed  in  their  origins. 
Furthermore,  they  were  primarily  defined  by  a  different  type  ("Internal",  "External")  of 
effective  IVs.  Relative  Phase  trends  were  primarily  defined  by  an  "External"  IV  (Cl  ef¬ 
fect  in  both  auditory  and  visual  trends).  Relative  magnitude  trends  were  primarily  de¬ 
fined  by  "Internal"  IVs  (SF  and  SI  effects  for  the  auditory  and  visual  trends, 
respectively).  These  observations  led  to  the  following  implications. 

9.2.4  Modeling  of  the  auditory-visual  interaction  process 

Cross-modality  averaged  delay/latency  estimations  of  Beta  and  Theta  MF  region 
CEPs  were  found  to  be  35/65  msec  and  55/230  msec,  respectively,  suggesting  different 
cortical  sources  of  these  potentials.  Since  phase  variations  induced  by  the  cross- 
modality  stimulus  were  unaffected  by  the  MF  region  of  the  intra-modality  stimulus  and 
were  uncoupled  from  the  magnitude  variations,  a  serially-connected,  'Beta  -"Theta" 
cortical  afferent  system  model  could  be  deduced.  In  this  model,  the  magnitude  variation 
dependency  on  intra-modality  parameters  (i.e.  the  audio  MF  region)  could  imply  the 
existence  of  selective,  inter-sensory  inhibitory  pathways  (negative  feedbacks). 

In  this  study,  such  an  AxV-related  CEP  magnitude  inhibition  was  associated  with  the 
shorter-latency,  Beta  MF  region  (FIGURES  III  8.10,  8.12).  It  is  interesting  to  speculate 
on  the  source  of  this  cross-sensory  inhibition.  Ciganek  (1966)  recorded  TEPs  in  response 
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to  click  and  flash  bimodal  stimuli  with  inter-modality  intervals  of  40-250  msec  (flash  lag). 
A  consistent  trend  of  reduced  visual  TEP  magnitude  components  at  70-80  msec  latency 
was  associated  with  long  (150-250  msec)  intcr-modality  intervals.  Morrell  (196S)  re¬ 
corded  TEPs  in  response  to  attended-to  flash  and  click  bimodal  stimuli  with  intcr- 
modality  intervals  of  20-120  msec  (click  lag).  Reduced  bisensory  TEP  magnitudes  at 
45-130  msec  latency  were  associated  with  long  inter-modality  intervals  (longer  than 
70-120  msec).  These  results  may  suggest  that  inhibition  of  "early"  latency  magnitude 
components  was  prompted  by  the  cross-sensory,  "late"  evoked  response  region.  If  CEP 
and  TEP  studies  arc  comparable,  these  "early"  and  "late"  latencies  could  correspond  to 
the  estimated  cross-modality  averaged  latencies  of  Beta  (60-70  msec)  and  Theta  (200-240 
msec)  MF  regions,  respectively.  Consequently,  the  inhibitory  effect  on  Beta  ME  mag¬ 
nitudes  observed  in  this  study  might  have  originated  from  the  cross-modality,  late 
latency,  Theta  M  F  region  neural  system. 
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Part  IV:  Summary  and  Conclusions 


Chapter  10 

Summary  and  Conclusions 

This  chapter  summarizes  the  major  findings  and  conclusions  obtained  in  pursuing  the 
following  research  objectives:  developing  a  Continuous  Evoked  Potential  (CEP)  re¬ 
search  system  utilizing  the  phase-lock  technique,  characterizing  the  auditory  and  visual 
unisensory  responses  and  demonstrating  an  Auditory-Visual  Interaction  process  (AxV) 
on  a  selective  stimulus  parameter-space.  The  specific  AxV  aspect  investigated  here  was 
the  dependency  of  single  sensory  channel  CEP  variations  on  cross-  or  inter-modality 
stimulus  parameters. 

1.  Detectable  stimulus-exclusive  CEPs  were  obtained  under  the  following  experiment 
conditions:  68-108  dB  SPL  monaurally  administered  audio  stimuli,  4-44  lux 
binocularly  viewed  visual  stimuli,  controlled  subject  attention,  Cz-A  1  (auditory)  and 
Oz-Al  (visual)  recording  sites  and  1-10  second  Lock-In  Amplifier  (LIA)  Time- 
Constant  (TC)  (section  I  3.4).  The  CEP  Signal/Noise  Ratio  (SNR)  must  be  further 
enhanced  to  allow  detection  of  CEPs,  evoked  by  more  tolerable  and  applicable 
stimuli  level,  within  a  similar  1-10  second  acquisition  time. 
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2.  Within  the  5-61  Hz  Modulation  Frequency  (MF)  range,  magnitude  peaks  (magni¬ 
tude  range  of  0.3-3  microvolt  RMS,  visual  higher  than  auditory)  were  found  at  the 
following  MF  regions:  5-6  Hz  ("Theta",  most  prominent  peak  in  both  modalities), 
10.5  Hz  ("Alpha",  auditory  CEP  only),  16-25  Hz  ("Beta",  both  modalities)  and  50 
Hz  ("40  Hz",  auditory  CEP  only)  (FIGURE  II  4.4).  The  visual  CEP  magnitude  data 
of  Alpha  and  Theta  MF  regions  contradicted  the  results  of  other  investigators.  The 
differences  were  attributed  to  the  effects  of  controlled  subject  attention  and  the 
driven-reference  CEP  recording  method  utilized  in  this  study  (section  II  4.4.4). 

3.  Except  for  the  Theta  frequency  region,  similar  EEG  background  activity  (0.5-2 
microvolt  RMS,  predominantly  Alpha)  was  obtained  from  both  auditory  and  visual 
recording  sites.  The  higher  auditory  Theta  EEG  level  was  attributed  to  myogenic 
and  other  sources  (FIGURE  H  4.2).  The  dissimilarity  between  the  EEG  spectrums 
and  the  CEP  Modulation  Transfer  Functions  (MTFs)  suggested  the  origin  or  source 
dissociation  of  these  two  phenomena. 

4.  Theta  and  Beta  MF  regions  were  further  investigated  in  the  DISC,  INT  and  ATT 
experiments  and  employed  in  the  Ax V  experiment.  CEPs  recorded  from  these  M  F 
regions  were  characterized  by  the  following:  consistent  responses  (magnitude  peaks 
were  detected  in  75-85%  of  the  Modulation  Transfer  Function  (MTF)  records, 
FIGURE  II  4.11);  evoked  responses  (CEP  sources  could  not  be  modeled  by  the 
EEG  equivalent  filters,  FIGURES  II  4.2,  4.4);  modality  compatible  (similar  magni¬ 
tude  and  latency  MF  regions  were  identified  in  both  modalities,  FIGURE  II  4.4, 
4.9);  response  repeatability  (a  similar  Beta/Theta  CEP  magnitude  ratio  was  ob¬ 
tained  in  all  the  experiments);  physiologically  interprctablc  (meaningful  CEP  delay 
and  latency  estimations  were  obtained). 
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5.  CEPs  recorded  in  response  to  100%  modulation  depth,  sinusoidally  amplitude- 
modulated  signals  were  predominantly  composed  of  the  fundamental  components 
(average  2nd/ fundamental  harmonic  ratio  of  0.43)  (TABLE  11  5.2).  lhis  harmonic 
ratio  was  obtained  at  the  highest  stimulus  intensity  ("0"  dB)  where  the  CEP  funda¬ 
mental  magnitudes  were  saturated  (FIGURE  II  5.3).  Based  on  these  findings,  it  was 
concluded  that  the  Beta  and  Theta  NlF  region  responses  were  predominantly  linear. 

6.  Linear  models  of  the  Theta,  Alpha,  Beta  and  "40  Hz"  MF  regions  (quadratic  BPL 
or  LPF  models)  were  fitted  to  the  CEP  magnitude  data.  CEP  delays  and  latencies 
were  estimated  from  the  phase  plots  of  the  models  and  CEP  data  in  the  vicinity  of 
the  transfer  function  peak  frequencies.  These  models  served  to  validate  the  physio¬ 
logical  origins  of  the  CEPs.  In  general,  the  higher  magnitude  visual  CLPs  were 
characterized  by  lower  "Q"  factor  models,  and  longer  and  more  reliable  estimated 
delays  (FIGURE  II  4.10,  TABLE  tl  4.2). 

7.  Delay  and  latency  estimations  of  Beta  MF  region  CEPs  were  30-40  msec  and  60-70 
msec,  respectively,  suggesting  the  "secondary-sensory"  cortical  areas  as  the  origin 
of  these  potentials.  Theta  MF  region  delay  and  latency  estimations  were  50-60  msec 
and  200-240  msec,  respectively,  suggesting  multiple,  secondary  cortical  area  sources 
(section  II  4.4.6).  These  results,  indicating  the  cortical  level  as  the  plausible  origins 
of  the  CEPs,  generally  agrees  with  previously  published  results.  Since  the  sensory 
integration  phenomena  is  more  likely  to  be  discovered  at  late  evoked  potentials 
(50-500  msec  latency  range),  the  CEPs  recorded  in  this  study  might  possibly  reflect 
an  AxV  process. 

8.  Attention  allocation  significantly  affected  only  the  visual  CEPs.  Attended-to  stimuli 
induced  higher  magnitude  CEPs  (FIGURE  II  5.4,  TABLE  II  5.4).  The 
inconclusivcness  of  the  auditory  results  may  be  attributed  to  the  well-saturated 
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auditory  CEP  magnitudes,  even  at  a  -12  dB  stimulus  intensity  level  (see  FIGURE 

11  5.3). 

9.  Regarding  the  intra-modality  CEP  variability,  the  visual  response  was  characterized 
by  higher  CEP  absolute  magnitude  and  lower  N'L  absolute  magnitude  (FIGURES 
II  4.4,  III  8.3),  higher  SNR  (section  II  4.4.2  and  FIGURE  111  8.3,  legend),  higher 
inter-subject  variability  (TABLE  II  4.1,  SD  measure;  TABLES  III  8.3,  8.5  NAME 
effect;  FIGURES  III  8.3,  8.5)  and  lower  intra-subject  magnitude  variability  (FIG¬ 
URE  II  5.2).  All  the  above  "features"  should  have,  retrospectively,  made  the  visual 
CEP  a  "preferred"  vehicle  for  the  AxV  phenomenon.  Nevertheless,  both  sensory 
channels  exhibited  an  AxV  process  (difference  between  unimodal  and  bimodal  re¬ 
sponses). 

10.  Variations  of  CEPs  recorded  in  one  modality  could  reliably  and  physiologically  be 
attributed  to  the  cross-modal  sensory  channel  activity  (section  III  9.1),  validating 
the  AxV  experiment  ad-hoc  assumptions  (section  I  3.2).  Within  the  investigated 
parameter-space,  maximal  bimodal-unimodal  CEP  differences  of  5.5  dB  magnitude- 
gain  and  40  degrees  phase-shift  were  attributed  to  the  AxV  phenomena  (section  III 
9.2.1,  FIGURES  III  8.10,  8.12). 

1 1.  CEP  magnitude  inhibition  and  phase  lag,  attributed  to  the  AxV  process,  were  gen¬ 
erally  associated  with,  and  induced  by  the  short-latency,  Beta  MF  regions.  These 
inter-sensory  mutual  decrements  were  not,  however,  associated  with  a  common 
paramctcr-space  origin  (the  enhanced  trend  in  FIGURES  III  8.10,  8.12).  This 
phenomena  could  be  interpreted  as  a  "lateral  inhibition"  between  sensory  primary 
cortical  areas. 
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12.  In  general,  trend  peaks  of  relative  magnitude  and  phase  data  (bimodal-unimodal 
CEP  response  difference)  were  not  concurrently  induced  or  constructively  comple¬ 
mented  (TABLES  111  8.3,  8.4;  FIGURES  III  8.10,  8.12;  Section  III  9.2.3).  Magni¬ 
tude  data  was  primarily  influenced  by  "Internal"  parameters  (intra-modalitv 
Stimulus  Intensity  or  MF)  and  phase  data  by  "External"  parameters  (Cross-modality 
stimulus  Intensity).  Surprisingly,  subject  attention  allocation  did  not  play  a  major 
role. 

13.  Cross-modality  averaged  dclay/latency  estimations  of  Beta  and  Theta  V1F  region 
CEPs  were  found  to  be  35/65  msec  and  55/230  msec,  respectively,  suggesting  differ¬ 
ent  cortical  sources  of  these  potentials.  Since  phase  variations  induced  by  the 
cross-modality  stimulus  were  unaffected  by  the  MF  region  of  the  intra-modality 
stimulus  and  were  uncoupled  from  the  magnitude  variations,  a  serially-connected, 
"Bcta"-"Thcta"  cortical  afferent  system  model  could  be  deduced.  In  this  model,  the 
magnitude  variation  dependency  on  intra-modality  parameters  could  imply  the  ex¬ 
istence  of  selective,  inter-sensory  inhibitory  pathways  (negative  feedbacks). 

The  motivation  for  this  research  was  to  develop  an  evoked  potential  methodology  for 
non-invasively  monitoring  the  auditory  and  visual  sensory  channel  engagement  and 
interaction  (AxV)  in  humans.  Parameter-space  characterization  of  the  AxV  process  has 
never  been  attempted  before.  Results  obtained  in  this  investigatory  study  can  be  uti¬ 
lized  in  many  existing  disciplines  and  promote  new  research  directions:  providing  further 
understanding  of  human  sensory  channel  transfer  functions  and  sensory  information  in¬ 
tegration,  investigating  the  unaccounted  for  intra-subject  unisensory  Event  Related 
Cortical  Potential  (ERCP)  variability  by  applying  an  exhaustive  multisensory  input 
stimulus,  facilitating  the  effectiveness  of  audio-visual  trainers,  monitoring  sensory  chan¬ 
nel  engagement  and  alerting  against  overloading  tasks  that  might  deteriorate  human- 
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operator  performance,  and  designing  more  effective  and  efficient  man-machine 
communication  channels. 

This  study  is  a  necessary  first  step  in  fulfilling  the  above  desired  goals.  Presently,  an 
integrative  sensory-system  model  needs  to  be  formulated  and  the  dependency  between 
evoked  potential  and  behavior  outcomes  must  be  established. 
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SUMMARY  OF  INDIVIDUAL  RESPONSES 
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4.0  6.3  10.0  15. B  25.1  39.8  63.1 

MODULATION  FREQUENCY  IN  HZ 


-0.0 


FIGURE  H  4.11  :  RECORDED  FROM  CZ-A1  (AUDIO)  AND  0Z-A1  (VISUAL)  SITES.  CEP(C). 

EEG(E),  AND  NOISE  LEVEL  (DOTTED  LINE)  MAGNITUDES  IN  DB.  CEP 
PHASE(P)  IN  RADIANS,  LATENCT(L)  IN  SEC  AND  UTXNCT  RELIABILITY 
(R,  1  IS  THE  HIGHEST,  ONLY  R>0.71  VALUES  ARE  DISPLAYED). 

L  (REGRESSION  C0XFF/6.Z8)  AND  R  (CORRELATION  COETT)  VALUES 
WERE  ESTIMATED  PROM  A  6  PHASE-PODfT-WIDE  SLIDING  WINDOW. 
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SUMMARY  OF  INDIVIDUAL  RESPONSES 

NAME-AM  SIG-VISUAL 


4.0  6.3  10.0  15.8  25.1  39. B  63.1 


MODULATION  FREQUENCY  IN  HZ 

FIGURE  □  4.11 :  RECORDED  FROM  CZ-A1  (AUDIO)  AND  OZ-A1  (VISUAL)  SITES.  CEP(C), 
EEG(E),  AND  NOISE  LEVEL  (DOTTED  LINE)  MAGNITUDES  IN  DB.  CEP 
PHASE  (P)  IN  RADIANS,  LATENCT(L)  IN  SEC  AND  LATENCY  RELIABILITY 
(R,  1  IS  THE  HIGHEST,  ONLY  R>0.71  VALUES  ARE  DISPLAYED). 

L  (REGRESSION  COEFF/B.E9)  AND  R  (CORRELATION  COEFT)  VALUES 
WERE  ESTIMATED  FROM  A  6  PHASE-POINT-WIDE  SLIDING  WINDOW. 
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SUMMARY  OF  INDIVIDUAL  RESPONSES 

NMIE-CR  SIG-AUOIO 


FIGURE  D  4.11  :  RECORDED  FROM  CZ-A1  (AUDIO)  AND  0Z-A1  (VISUAL)  SITES.  CKP(C), 
EEC(E),  AND  NOISE  LEVEL  (DOTTED  LINE)  MAGNITUDES  IN  DB.  CEP 
PHASE(P)  IN  RADIANS,  LATENCY(L)  IN  SEC  AND  LATENCY  RELIABILITY 
(R,  1  IS  THE  HIGHEST,  ONLY  R>0.71  VALUES  ARE  DISPLAYED). 

L  (REGRESSION  COEFF/6.29)  AND  R  (CORRELATION  COEFF)  VALUES 
WERE  ESTIMATED  FROM  A  S  PHASE-POINT-WIDE  SLIDING  WINDOW. 
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4.0  6.3  10.0  IS . B  25.1  39. B  63.1 

UODULAT ION  FREQUENCY  IN  HZ 


FIGURE  A  4.11  :  RECORDED  FROM  C2-A1  (AUDIO)  AND  OZ-A1  (VISUAL)  STIRS.  CEP(C), 
EEG(E),  AND  NOSE  LEVEL  (DOTTED  LINE)  MAGNITUDES  IN  DB.  CEP 
PHASE  (P)  IN  RADIANS,  LA1XNCY(L)  IN  SIX!  AND  LATENCY  RELIABILITY 
(R,  1  IS  THE  HIGHEST,  ONLY  R>0.71  VALUES  ARE  DISPLAYED). 

L  (REGRESSION  COEFT/6.29)  AND  R  (CORRELATION  COETF)  VALUES 
WERE  ESTIMATED  FROM  A  6  PHASE-POINT-WIDE  SLIDING  WINDOW. 
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SUMMARY  OF  INDIVIDUAL  RESPONSES 

NAME-EW  SIG-AUDIO 


FIGURE  II  i.11  :  RECORDED  FROM  CZ-A1  (AUDIO)  AND  OZ-A1  (VISUAL)  SITES.  CEP(C), 
EEG(E),  AND  NOISE  LEVEL  (DOTTED  LINE)  MAGNITUDES  IN  DB.  CEP 
PHASE(P)  IN  RADIANS,  LATBTCY(L)  IN  SEC  AND  UTXNCV  RELIABILITY 
(R.  1  IS  THE  HIGHEST,  ONLY  R>0.71  VALUES  ARE  DISPLAYED). 

L  (REGRESSION  CQEFT/6.28)  AND  R  (CORRELATION  COETF)  VALUES 
WERE  ESTIMATED  FROM  A  6  PHASE-POINT-WIDE  SLIDING  WINDOW. 
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4.0  6.3  10.0  15. B  25.1  39. B  63.1 

MODULATION  FREQUENCY  IN  HZ 


HOURS  0  4.11  :  RECORDED  PROM  CZ-A1  (AUDIO)  AND  OZ-A1  (VISUAL)  SITES.  CEP(C), 
EEG<E),  AND  NOISE  LEVEL  (DOTTED  UNE)  MAGNITUDES  IN  DB.  CEP 
PHASE  (P)  IN  RADIANS,  LATENCY(L)  IN  SEC  AND  LATENCY  RELIABILITY 
(R,  1  IS  THE  HIGHEST,  ONLY  R>0.7i  VALUES  ARE  DISPLAYED). 

L  (REGRESSION  COEVT/8.S9)  AND  R  (CORRELATION  COEFT)  VALUES 
WERE  ESTOIATED  PROM  A  6  PHASE-POINT- WIDE  SLIDING  WINDOW. 
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SUMMARY  OF  INDIVIDUAL  RESPONSES 

NAME-KC  SIG-AUDIO 


FIGURE  n  4.11  :  RECORDED  FROM  CZ-A1  (AUDIO)  AND  OZ-A1  (VISUAL)  SITES.  CEP(C). 

EEC(E),  AND  NOISE  LEVEL  (DOTTED  LINE)  MAGNITUDES  IN  DB.  CEP 
PHASE(P)  IN  RADIANS,  LATENCY(L)  IN  SEC  AND  LATENCY  RELIABILITY 
(R,  1  IS  THE  HIGHEST.  ONLY  R>0.71  VALUES  ARE  DISPUYED). 

L  (REGRESSION  C0E7T/6.28)  AND  R  (CORRELATION  COEIT)  VALUES 
WERE  ESTIMATED  TROM  A  S  PHASE-POINT-WIDI  SLIDING  WINDOW. 
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SUMMARY  OF  INDIVIDUAL  RESPONSES 

NAME=KC  S I G=V I SUAL 


4.0  6.3  10.0  15. B  25.1  39. B  63.1 


MODULATION  FREQUENCY  IN  HZ 

FIGURE  0  4.11  :  RECORDED  FROM  CZ-A1  (AUDIO)  AND  OZ-A1  (VISUAL)  SITES.  CEP(C), 
EEG(E),  AND  NOISE  LEVEL  (DOTTED  LINE)  MAGNITUDES  IN  DB.  CEP 
PHASE  (P)  IN  RADIANS,  LATENCY(L)  IN  SEC  AND  LATENCY  RELIABILITY 
(R,  1  IS  THE  HIGHEST,  ONLY  R>0.71  VALUES  ARE  DISPLAYED). 

L  (REGRESSION  COEFF/B.Zfl)  AND  R  (CORRELATION  COEFT)  VALUES 
WERE  ESTIMATED  FROM  A  B  PHASE-POINT-WIDE  SLIDING  WINDOW. 
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SUMMARY  OF  INDIVIDUAL  RESPONSES 

NAME-MR  SIG-AUDIO 


FIGURE  D  4.11  :  RECORDED  FROM  CZ-A1  (AUDIO)  AND  0Z-A1  (VISUAL)  SITES.  CEP(C). 

EEC(E),  AND  NOISE  LEVEL  (DOITED  LINE)  MAGNITUDES  IN  DB.  CEP 
PHASE(P)  IN  RADIANS.  LATENCY(L)  IN  SEC  AND  LATENCY  RELIABILITY 
(R,  1  IS  THE  HIGHEST,  ONLY  R>0.71  VALUES  ARE  DISPLAYED). 

L  (REGRESSION  C0EFF/8.2V)  AND  R  (CORRELATION  COETT)  VALUES 
WERE  ESTIMATED  FROM  A  S  PHASE-POINT-WIDE  SLIDING  WINDOW. 
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SUMMARY  OF  INDIVIDUAL  RESPONSES 

name=mr  sig=visual 


FIGURE  n  4.11  :  RECORDED  FROM  CZ-A1  (AUDIO)  AND  0Z-A1  (VISUAL)  SITES.  CEP(C), 
EEG(E),  AND  NOISE  LEVEL  (DOTTED  LINE)  MAGNITUDES  IN  DB.  CEP 
PHASE(P)  IN  RADIANS,  LATENCY(L)  IN  SEC  AND  LATENCY  RELIABILITY 
(R.  1  IS  THE  HIGHEST,  ONLY  R>0.71  VALUES  ARE  DISPLAYED). 

L  (REGRESSION  COEFT/B.29)  AND  R  (CORRELATION  COEFT)  VALUES 
WERE  ESTIMATED  FROM  A  5  PHASE-POINT-WIDE  SLIDING  WINDOW. 
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SUMMARY  OF  INDIVIDUAL  RESPONSES 

NAME=PM  SIG=AUDIO 


FIGURE  U  A.U  :  RECORDED  FROM  CZ-A1  (AUDIO)  AND  0Z-A1  (VISUAL)  SUES.  CEP(C), 
EEG(E).  AND  NOISE  LEVEL  (DOTTED  LINE)  MAGNITUDES  IN  DB.  CEP 
PHASE(P)  IN  RADIANS,  LATENCY(L)  IN  SEC  AND  LATENCY  RELIABILITY 
(R,  1  IS  THE  HIGHEST,  ONLY  R>0.?1  VALUES  ARE  DISPLAYED). 

L  (REGRESSION  COEFF/6.29)  AND  R  (CORRELATION  COEFF)  VALUES 
WERE  ESTIMATED  FROM  A  fi  PHASE -POINT- WIDE  SLIDING  WINDOW. 
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SUMMARY  OF  INDIVIDUAL  RESPONSES 

NAUE=PU  S I G=V I SUAL 


FIGUKE  n  4.11  :  RECORDED  FROM  CZ-A1  (AUDIO)  AND  0Z-A1  (VISUAL)  SITES.  CEP(C), 
EEG(E),  AND  NOISE  LEVEL  (DOTTED  LINE)  MAGNITUDES  IN  DB.  CEP 
PHASE(P)  IN  RADIANS,  LATENCY(L)  IN  SEC  AND  LATENCY  RELIABILITY 
(R,  l  IS  THE  HIGHEST,  ONLY  R>0.71  VALUES  ARE  DISPLAYED). 

L  (REGRESSION  COEFF/6.2B)  AND  R  (CORRELATION  COETT)  VALUES 
WERE  ESTIMATED  FROM  A  5  PHASE-POINT-WIDE  SLIDING  WINDOW. 
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SUMMARY  OF  INDIVIDUAL  RESPONSES 

NAME-RT  SIG-AUDIO 


FIGURE  n  4.11  :  RECORDED  FROM  CZ-A1  (AUDIO)  AND  OZ-A1  (VISUAL)  SITES.  CEP(C), 
EEC(E),  AND  NOISE  LEVEL  (DOTTED  LINE)  MAGNITUDES  IN  DB.  CEP 
PHASE(P)  IN  RADIANS,  LATENCY(L)  IN  SEC  AND  LATENCY  RELIABILITY 
(R,  1  IS  THE  HIGHEST,  ONLY  R>0.71  VALUES  ARE  DISPLAYED). 

L  (REGRESSION  COEFT/B.CT)  AND  R  (CORRELATION  COETF)  VALUES 
WERE  ESTIMATED  FROM  A  5  PHASE-POINT- WIDE  SLIDING  WINDOW. 
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SUMMARY  OF  INDIVIDUAL  RESPONSES 
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4.0  6.3  10.0  15. B  25.1  39. 8  63.1 

MODULATION  FREQUENCY  IN  HZ 

FIGURE  H  4.11  :  RECORDED  FROM  CZ-A1  (AUDIO)  AND  OZ-A1  (VISUAL)  SITES.  CEP(C), 
EEG(E),  AND  NOISE  LEVEL  (DOTTED  LINE)  MAGNITUDES  IN  DB.  CEP 
PHASE(P)  IN  RADIANS,  LATENCY(L)  IN  SEC  AND  LATENCY  RELIABILITY 
(R,  1  IS  THE  HIGHEST,  ONLY  R>0.71  VALUES  ARE  DISPLAYED). 

L  (REGRESSION  COEFF/6.SB)  AND  R  (CORRELATION  COEFF)  VALUES 
WERE  ESTIMATED  FROM  A  6  PHASE-POINT-WIDE  SLIDING  WINDOW. 
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SUMMARY  OF  INDIVIDUAL  RESPONSES 

NAME-SC  SIG*AUDI0 
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4.0  6.3  10.0  15.8  25.1  39.8  63.1 

MODULATION  FREQUENCY  IN  HZ 
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FIGURE  n  4.11  :  RECORDED  FROM  CZ-A1  (AUDIO)  AND  0Z-A1  (VISUAL)  SITES.  CEP(C), 
EEG(E),  AND  NOISE  LEVEL  (DOTTED  UNI)  MAGNITUDES  IN  DB.  CEP 
PHASE(P)  IN  RADIANS,  LATENCY(L)  IN  SEC  AND  LATENCY  RELIABILITY 
(R,  1  IS  THE  HIGHEST,  ONLY  R>0.71  VALUES  ARE  DISPLAYED). 

L  (REGRESSION  COEFF/6.29)  AND  R  (CORRELATION  COEFF)  VALUES 
WERE  ESTIMATED  FROM  A  5  PHASE-POINT-WIDE  SLIDING  WINDOW. 
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SUMMARY  OF  INDIVIDUAL  RESPONSES 

NAME-SC  SIG-VISUAL 


FIGURE  n  4.11  :  RECORDED  IRON  CZ-A1  (AUDIO)  AND  OZ-Al  (VISUAL)  STIES.  CEP(C), 
EEC(£),  AND  NOISE  LEVEL  (DOTTED  LINE)  MAGNITUDES  IN  DB.  CEP 
PHASE  (P)  IN  RADIANS,  LATENCT(L)  IN  SEC  AND  LATENCY  RELIABILITY 
(R,  1  IS  THE  HIGHEST,  ONLY  R>0.71  VALUES  ARE  DISPLAYED). 

L  (REGRESSION  CODT/6.28)  AND  R  (CORRELATION  COEIT)  VALUES 
WERE  ESTIMATED  FROM  A  5  PHASE-POINT-WIDE  SLIDING  WINDOW. 


-174- 


-n-nmoo  zo - i»rmx»00 


SUMMARY  OF  INDIVIDUAL  RESPONSES 

NAUE-TN  SIG-AUDIO 


FIGURE  II  4.11  :  RECORDED  FROM  CZ-A1  (AUDIO)  AND  OZ-A1  (VISUAL)  SITES.  CEP(C), 

EEG(E),  and  noise  level  (dotted  line)  magnitudes  in  db.  cep 

PHASE(P)  IN  RADIANS,  LATENCY(L)  IN  SEC  AND  LATENCY  RELIABILITY 
(R,  1  IS  THE  HIGHEST,  ONLY  R>0.71  VALUES  ARE  DISPLAYED). 

L  (REGRESSION  COETT/6.Z9)  AND  R  (CORRELATION  COEFT)  VALUES 
WERE  ESTIMATED  FROM  A  5  PHASE-POINT-WIDE  SLIDING  WINDOW. 
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MODULATION  FREQUENCY  IN  HZ 


FIGURE  n  4.11  :  KI CORDED  FROM  CZ-Al  (AUDIO)  AND  0Z-A1  (VISUAL)  SITES.  CEP(C), 
EtC(X),  AND  NOISE  LEVEL  (DOTTED  LINE)  MAGNITUDES  IN  DB.  CEP 
PHASE  (P)  IN  RADIANS,  1ATENCT(L)  IN  SEC  AND  LATENCY  RELIABILITY 
(R,  1  IS  THE  HIGHEST,  ONLY  RX1.71  VALUES  ARE  DISPLAYED). 

L  (REGRESSION  COEFF/6.29)  AND  R  (CORRELATION  COEFF)  VALUES 
WERE  ESTIMATED  FROM  A  6  PHASE-POINT -WIDE  SLIDING  WINDOW. 
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Appendix  B.  Alphabetic  List  of  Abbreviations 


A 

AM 

ANOVA 

AP 

ATI' 

AVD 

AxV 

B! 

BPF 

BW 

CEP 

CF 

CFF 

CJ 

CMRR 

CV 

DA 

DAC 

DISC 

ECG 

EEG 

EMG 

EP 

ERCP 

FM 

FREQ 

GATE 

HL 

HM 

HPF 

I  Ml 

INI 

IPI 

1SF 

IV 

LI  A 

EPF 

M 

MANOV \ 

MF 


auditor^’ 

amplitude  modulat  ed/ion 
analysis  of  variance 
action  potential 
attention  or  attended-to 
audio  visual  delay 
auditory-visual  interaction 
bimodal 
band-pass  filter 
bandwidth 

continuous  evoked  potential 

cross-modal  stimulus  modulation  frequency 

critical  flicker  frequency 

cross-modal  stimulus  intensity 

common-mode  rejection  ratio 

coefficient  of  variability 

differential  amplifier 

digital-to-analog  converter 

discrete 

electrocardiogram 

electroencephalogram 

electromyogram 

evoked  potential 

event  related  cortical  potential 

frequency  modulat  cd/ion 

frequency 

stimulus  on/off  signal 
hearing  level 
"home  made"  device 
high-pass  filter 
inter-modality  interval 
intensity 

inter-pattern  interval 
inter-stimulus  interval 
independent  variable 
lock-in  amplifier 
low-pass  filter 

absolute,  compensated  or  relative  magnitude 
multiple  analysis  of  variance 
modulation  frequency 
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MOD 

MSE 

MTF 

N 

NAME 

NAT 

NL 

NS 

P 

Q 

R 

REF 

REF* 

RMS 

RMSE 

RT 

S 

SD 

SF 

SI 

SE 

SIG 

SL 

SNR 

SPL 

TC 

TEP 

UNI 

V 

VS 


stimulus  number  of  modalities 
mean  square  error 
modulation  transfer  function 
number  of  elements 
subject  code 
not-attended-to 
magnitude  noise  level 
not  significant 

probability  or  absolute,  compensated  or  relative  phase 

quality  (filter  sharpness)  factor 

correlation  coefficient 

reference  MF  signal 

REF  +  the  ac-coupled  GATE  signal 

root  mean  square 

root  mean  square  error 

reaction  time 

stimulus  duration 

standard  deviation 

intra-modal  stimulus  modulation  frequency 
intra-modal  stimulus  intensity 
standard  error 

stimulus  and  CEP  modality  or  electrical  signal 

sound  level 

signal-to-noise  ratio 

sound  pressure  level 

time  constant 

transient  evoked  potential 

unimodal 

visual 

visual  stimulator 


